# Phygital Intelligence Tongyue Sun · Philip F. Yuan Editors

**Computational Design and Robotic Fabrication**

Chao Yan · Hua Chai ·

Proceedings of the 5th International Conference on Computational Design and Robotic Fabrication (CDRF 2023)

# **Computational Design and Robotic Fabrication**

Series Editor Philip F. Yuan, *Tongji University, Shanghai, China* This open access book series includes compilations of selected papers from the International Conference on Computational Design and Robotic Fabrication. The books focus on novel techniques for computational design and robotic fabrication. It not only aims at the most recent research results from the key scholars in the computational design and robotic fabrication, but also offers an in-depth examination of intelligence in design and construction industry, referring to the invention and application of machine intelligence in architecture.The proceedings in this series are related with Sustainable Development Goals 11 (Sustainable Cities & Communities) and 9 (Industry Innovation & Infrastructure).

The contents make valuable contributions to academic researchers, designers, and engineers in the industry. As well, readers will encounter new ideas about understanding intelligence in architecture.

Chao Yan · Hua Chai · Tongyue Sun · Philip F. Yuan Editors

# Phygital Intelligence

Proceedings of the 5th International Conference on Computational Design and Robotic Fabrication (CDRF 2023)

*Editors* Chao Yan College of Architecture and Urban Planning Tongji University Shanghai, China

Tongyue Sun College of Architecture and Urban Planning Tongji University Shanghai, China

Hua Chai College of Architecture and Urban Planning Tongji University Shanghai, China

Philip F. Yuan College of Architecture and Urban Planning Tongji University Shanghai, China

ISSN 2731-9040 ISSN 2731-9059 (electronic) Computational Design and Robotic Fabrication ISBN 978-981-99-8404-6 ISBN 978-981-99-8405-3 (eBook) https://doi.org/10.1007/978-981-99-8405-3

This book is funded by National Science Foundation of China-shenzhen Joint Fund (No. U1913606), Science and Technology Innovation Plan of Shanghai Science and Technology Commission (No. 21DZ1204500).

© The Editor(s) (if applicable) and The Author(s) 2024. This book is an open access publication.

**Open Access** This book is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this book are included in the book's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the book's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd. The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore

Paper in this product is recyclable.

# **Preface**

Architecture is currently facing a physical-digital era that requires a re-consideration and re-assembly of traditional concepts. From digital twins to meta-universes, a series of technological trends have projected the futures of architectural production and inhabitation on the planetary scale. From global warming, to the pandemic, a series of social and ecological upheavals have also unveiled the urgency to re-define the relationship between globalization and locality.

To propose new methodologies and new tools within the architectural ontology of the phygital future, we need to fundamentally rethink the boundary between the virtual and the real. What is the future of the relationship between physical space and virtual intelligent technologies? Will they coexist in parallel, or will they integrate and synergize together?

To meet the challenges of the current existential crisis, we also need to fundamentally rethink the relationship between architecture, nature, and human society in this phygital era. Is there a way in which the phygital nature of architectural production and inhabitation could provide a conceptually new model of sustainability as opposed to merely offering more engineering solutions?

As the theme of Architectural DigitalFUTURES 2023, Phygital Intelligence intends to promote a series of scholarly dialogues on these technological, social, and ecological transformations in architecture. The key objective is to re-define the term "digital twin" and to investigate the role of phygital intelligence in architectural production in all the transcended territories of the virtual and the real, the individual and the collective, as well as the global and the local.

Shanghai, China Chao Yan

Hua Chai Tongyue Sun Philip F. Yuan

# **Organization**

# **Committees**

# **Honorary Advisors**


### **Organization Committees**


# **Scientific Committees**


Assistant Prof. Daniel Bolojan Florida Atlantic University, USA Associate Prof. Dr. Matias Del Campo Prof. Brad Cantrell University of Virginia, USA Associate Prof. Dr. Tengwen Chang Associate Prof. Dr. Kristof Crolla University of Hong Kong, China Assistant Prof. Dr. Benjamin Dillenburger Marcus Farr, Tongji University China Melissa Goldman University of Virginia, USA Associate Prof. Dr. Hua Hao Southeast University, China Wanyu He Xkool, China Prof. Dr. Tim Heath University of Nottingham, UK Associate Prof. Dr. Weixin Huang Tsinghua University, China Prof. Dr. Guohua Ji Nanjing University, China Gene Ting-Chun Kao ETH Zurich, Switzerland

Dr. Guan Lee University College London, UK Associate Prof. Dr. Hyejin Lee Tongji University, China Prof. Dr. Biao Li Southeast University, China Prof. Dr. Linxue Li Tongji University, China Prof. Dr. Yujie Lu Tongji University, China Associate Prof. Dr. Peng Luo Tongji University, China Andrea Macruz Tongji University, China Assistant Prof. Sandra Manninger University of Michigan, USA Associate Prof. Dr. Wes McGee University of Michigan, USA Assistant Research Fellow, Dr. Xianchuan Meng

Serban Bodea University of Stuttgart, Germany Assistant Prof. Biayna Bogisian Florida International University in Miami, USA University of Michigan, USA

> National Yunlin University of Science and Technology, Taiwan, China ETH Zurich, Switzerland

Associate Prof. Dr. Yunsong Han Harbin Institute of Technology, China Associate Prof. Alvin Huang University of Southern California, USA Assistant Prof. Dr. Immanuel Koh Singapore University of Technology and Design, Singapore Prof. Dr. Neil Leach Tongji University, China Nanjing University, China Virginia Melnyk Tongji University, China/Clemson University,

USA Dr. Kris Mun University of Minnesota, USA Guvenc Ozel University of California, Los Angeles, USA Gilles Retsin University College London, UK Klaas de Rycke Bollinger + Grohmann, Germany Prof. Bob Sheil University College London, UK

Prof. Dr. Xing Shi Tongji University, China Prof. Dr. Miroslaw J. Skibniewski University of Maryland, USA Associate Prof. Dr. Roland Snooks RMIT University, Australia Associate Prof. Dr. Chengyu Sun Tongji University, China Prof. Dr. Kostas Terzidis Tongji University, China Associate Prof. Kathy Velikov University of Michigan, USA Tomas Vivanco Tongji University, China Dr. Xiang Wang Tongji University, China Prof. Dr. Makoto Sei Watanabe Tokyo City University, Japan Dr. Dylan Wood University of Stuttgart, Germany

Prof. Dr. Leiqing Xu Tongji University, China Prof. Dr. Weiguo Xu Tsinghua University, China Dr. Chao Yan Tongji University, China Associate Prof. Dr. Feng Yang Tongji University, China Associate Research Fellow, Dr. Jiawei Yao Prof. Dr. Philip F. Yuan Tongji University, China Prof. Dr. Xu Zhen Tianjin University, China

Satoru Sugihara Architectural Technology Laboratorial Venture Prof. Dr.-Ing. Oliver Tessmann Technische Universität Darmstadt, Germany Dr. Jing Wu Southern University of Science and Technology, China Prof. Michael Weinstock The Architectural Association, UK Tongji University, China Assistant Prof. Kaiho Yu University of Applied Arts Vienna, Austria

# **Contents**

#### **Computation and Formation**



Contents xiii




xiv Contents



# **Computation and Formation**

# **Deep Mining Authorship**

Sandra Manninger<sup>1</sup> and Matias del Campo2(B)

<sup>1</sup> NYIT (New York Institute of Technology), 16 W. 61St Street, Room 914 212.261.1741, New York, USA

smanning@nyit.edu

<sup>2</sup> Taubman College of Architecture and Urban Planning, University of Michigan, Ann Arbor,

USA

mdelc@umich.edu

**Abstract.** Considering the emerging field of architecture and artificial intelligence, it might be necessary to contemplate the remodeling of the concept of authorship entirely. The invention of authorship is a complex historical process that can be traced back to the emergence of print culture in Europe in the 15th century. Prior to this period, most literary and artistic works were created anonymously or attributed to collective or anonymous sources, such as folklore or religious traditions. However, with the rise of printing, texts became more easily reproducible and marketable, and there emerged a need for individual authors to take credit for their works. The notion of authorship was closely tied to the idea of originality and ownership, as authors sought to assert their exclusive rights to their works and to distinguish themselves from other writers. This was supported by the development of copyright law, which granted legal protection to authors and their works, and helped to establish a market for literary and artistic works. The idea of the author as a singular, autonomous figure gained further prominence in the 18th and 19th centuries, with the emergence of romanticism and the cult of the individual. This period saw the rise of the idea of the artist as a genius, whose works were the product of their own unique creativity and imagination. This idea was further reinforced by the rise of literary criticism, which focused on the interpretation and analysis of individual works and their authors. However, as Michel Foucault and other scholars have argued, the notion of authorship is not a universal or timeless concept, but rather a historically contingent and culturally specific one. Different societies ad cultures have different understandings of authorship, and these have shifted over time in response to changes in technology, culture, and social values. As it stands now, authorship in its traditional form can hardly be applied in a context where automated collaborations provide more than 50% of the generated material. This is true for multiple art fields. Visual Arts (Mario Klingemann, Sofia Crespo, Memo Atken, Ooouch, etc.), Music (Dadabots, YACHT, Holly Herndon), Literature, etc. Very soon this will also be true for Architecture. The consequence is also an entire rethinking of the concept of the sole genius. This notion, developed by German Romanticists in the early 19th century, is, in the current context of AI-assisted creativity, completely obsolete, as we are drawing from the genius of hundreds of thousands of artists and artworks in order to interrogate the latent space for unseen artistic opportunities. More akin to an archeological dig leading to the discovery of a next-generation jet fighter plane.

**Keywords:** Authorship · Agency · Genius · Artificial intelligence · Creativity · Theory

#### **1 Introduction—An Ontology of Authorship**

In this paper we present a position towards the concept of authorship in the context of the emergence of technologies and design techniques in architecture based on artificial intelligence (AI). In order to prepare the ground for a conversation on Authorship and AI, both of these fields need to be described and defined in the context of the debate in this paper.

The word "authorship" has been in use in the English language since at least the 16th century. The Oxford English Dictionary cites the first known use of the word in 1579, in a book titled "A Discourse of the Adventures passed by Master F. J. in his Voyage to the East Indies". The word "authorship" is derived from the Old French word "autor", meaning "creator or originator", which in turn comes from the Latin word "auctor", meaning "producer, father, or founder". The term "authorship" has since become an important concept in literature, academic writing, and other forms of creative expression such as architecture, referring to the act of creating or producing a written work and the recognition or attribution given to the individual or group who created the work.

The common understanding of 'Authorship' states: "*An author is 'the person who originated or gave existence to anything and whose authorship determines responsibility for what was created*".

Authorship in the context of this paper refers to the act of creating or producing artistic work, such as a book, painting, sculpture of design. It primarily refers to the recognition or attribution given to the individual or group who created the work. Authorship is important because it acknowledges and gives credit to the person or people who put in the time, effort, and creativity to produce the work.

In academic and research contexts, authorship can be a complex issue, as there are often multiple individuals who contribute to a piece of work, and determining who should be listed as an author can depend on a variety of factors such as the level of contribution, intellectual input, and responsibility. In such cases, authorship guidelines and protocols are usually established to ensure that appropriate credit is given to all those who have contributed to the work. The concept of authorship has been present for centuries and has been defined and discussed by various scholars, philosophers, and literary figures throughout history. However, the modern understanding of authorship and its legal implications can be traced back to the 18th and 19th centuries, when copyright laws were first introduced to protect the rights of authors and their intellectual property. In academia and research contexts, authorship guidelines and protocols have been established by various organizations and institutions, such as the International Committee of Medical Journal Editors (ICMJE), the Council of Science Editors (CSE), and the American Psychological Association (APA), to help researchers determine who should be listed as an author and ensure appropriate credit is given to all contributors.

In the arts, authorship can be a complex and subjective issue, as it involves questions about creativity, originality, and intellectual property. The definition of authorship in the arts can vary depending on the medium, genre, and cultural context of the work. In some cases, the artist or creator of a work may be considered the sole author, while in others, authorship may be shared or attributed to multiple individuals who contributed to the creation of the work. Some art forms, such as collaborative works, performance art, and conceptual art, may challenge traditional notions of authorship and emphasize the role of the audience or viewers in co-creating the work. Overall, the definition of authorship in the arts is often shaped by legal, cultural, and artistic considerations, and may be subject to interpretation and debate.

It is even more complex when it comes to architecture, as it involves the collaboration of multiple individuals and disciplines in the design and construction of a building or structure. The definition of authorship in architecture can vary depending on the specific project, the role and level of involvement of each participant, and the legal and cultural context of the work. Some architectural projects may be attributed to a single architect or design team, while others may involve multiple designers, engineers, contractors, and other stakeholders who contribute to the project in various ways. In some cases, authorship may be shared or collaborative, with each participant contributing their own unique ideas and expertise to the design process. Professional organizations such as the American Institute of Architects (AIA) and the Royal Institute of British Architects (RIBA) have established ethical standards and guidelines for architects and designers to ensure that appropriate credit is given to all contributors and that the integrity of the design process is maintained. However, authorship in architecture can still be a complex and debated issue, particularly in cases where disputes arise over ownership, attribution, or credit for a particular design or project.

### **2 Debate—Authorship, Authority and Language**

Authorship of language is a concept that relates to the origin and ownership of language, particularly in the context of its use in literature and other forms of creative expression. While language is a shared cultural resource that has evolved over time through the contributions of countless individuals and communities, authorship can be seen as a way of assigning ownership and attribution to specific uses of language in literature and other creative works. In the context of literature as well as in architecture, authorship of language or form can refer to the ways in which writers use language to create meaning and shape their work. This can include considerations such as the writer's style, voice, and use of literary devices, as well as broader questions about the cultural and historical context in which the work is created. All of which can be translated to the use of formal language in architecture design, which form a specific portion of the authorship of a design object. This being considered, the concept of authorship of language has been challenged by poststructuralist and postmodernist theories, which suggest that language itself is inherently unstable and fragmented, and that meaning is not fixed but rather constantly in flux. According to these theories, the author is not a single, unified figure who controls the meaning of a text, but rather a product of the social and cultural discourses in which they operate. Overall, authorship of language is a complex and contested concept that involves questions about the origins, ownership, and use of language in literature and other forms of creative expression.

Michel Foucault, a French philosopher and social theorist, was known for his influential work on the relationships between power, knowledge, and discourse. While Foucault did not specifically address the concept of authorship authority in his writings, his ideas on the nature of power and knowledge can be applied to the ways in which authorship is constructed and contested in various fields.

In his book "The Archaeology of Knowledge", Foucault argues that knowledge is not a static entity, but rather a dynamic, constantly evolving system of discursive practices that are shaped by power relations. He suggests that the way in which knowledge is produced and circulated is not neutral, but rather reflects the interests and perspectives of those in positions of power. This perspective can be applied to the issue of authorship authority, as it raises questions about who gets to decide what constitutes authoritative knowledge, and how these decisions are shaped by broader social, cultural, and political contexts. Foucault's work also highlights the ways in which authorship can be used as a tool of power, with certain individuals or groups being granted more authority or recognition for their work than others, based on their social status or institutional affiliations. Michel Foucault's position towards the author can be understood through his influential essay, "What is an Author?" In this essay, Foucault argues that the traditional understanding of the author as the singular, individual creator of a work is a relatively recent historical development, and that it is closely tied to notions of authorial intention, originality, and ownership. He suggests that these ideas emerged in the 18th and 19th centuries, and were closely tied to the rise of capitalism and the development of copyright law. Foucault critiques this traditional view of the author, arguing that it is overly individualistic and neglects the role of social, cultural, and historical factors in shaping the production and reception of a work. He suggests that works of literature are not created ex nihilo by individual authors, but are rather the result of a complex web of cultural and discursive practices that extend beyond the individual. Foucault also suggests that the notion of the author as an autonomous, transcendent figure is problematic because it encourages a form of interpretation that focuses solely on the author's intentions and meanings. He suggests that this approach neglects the role of readers and interpreters in shaping the meaning of a work, and that it limits the potential for creative and subversive readings. Overall, Foucault's position towards the author is complex and nuanced, and challenges traditional notions of authorship and creativity. He suggests that the author is not a singular, autonomous figure, but is rather shaped by a range of social, cultural, and historical factors, and that the meaning of a work is not fixed or determined by the author's intentions.

To round up the picture regarding the criticism of Poststructuralist thinkers, Roland Barthes cannot be ignored. "The Death of the Author" is a phrase coined by French literary critic Roland Barthes in his essay of the same name, first published in 1967. In this essay, Barthes argues that the traditional concept of the author as the sole originator and controller of the meaning of a text is a myth, and that the meaning of a text is instead created through the interactions between the reader and the text. Barthes asserts that the author is not a transcendent figure who imbues the text with meaning, but rather a historical and cultural construct that has been used to establish the authority of the writer and the ownership of the text. He suggests that the idea of the author as a single, unified source of meaning is a product of a particular historical and cultural moment, and that this concept has been used to control and limit the ways in which texts are interpreted and understood.

"The Death of the Author" has been influential in literary and cultural studies, as it challenges traditional notions of authorship, interpretation, and meaning-making, and emphasizes the role of the reader in co-creating the meaning of a text. It has also been criticized by some scholars who argue that it ignores the role of the author in shaping the text, and that it promotes an overly subjective and relativistic view of interpretation.

### **3 What is Authorship in the Age of AI?**

The ongoing debate of authorship and artificial intelligence (AI) revolves around questions considering the nature of creativity, the role of human agency, and the boundaries between human and machine-generated art and architecture. On the one side, proponents of AI-generated art argue that machines can produce creative works that are indistinguishable from those produced by humans, and that these works can challenge traditional notions of authorship and creativity. They also suggest that AI-generated art has the potential to democratize access to creative tools and to expand the boundaries of what is possible in art. On the other hand, critics of AI-generated art argue that machines lack the consciousness and intentionality of human creators, and that the notion of authorship is closely tied to human agency and the expression of individual identity. They also raise concerns about the potential for AI-generated art to displace human artists and to reinforce existing power structures in the art world. In addition to these philosophical debates, there are also practical and legal considerations around the ownership and copyright of AI-generated art, as well as questions about the ethical implications of using machines to produce art. Overall, the debate around authorship and AI-generated art is complex and multifaceted, and involves questions about creativity, agency, ownership, and ethics that are still being explored by artists, scholars, and policymakers.

To illustrate this point the authors would like rely on the example of the 'Portrait of Edmond de Belamy'.The painting was created by the French art collective Obvious in 2018, using a form of AI called Generative Adversarial Networks (GANs)24. The painting depicts a blurry, slightly distorted portrait of a fictional aristocrat named Edmond de Belamy, and was sold at Christie's for over \$430,000.

The creation of "Portrait of Edmond de Belamy" sparked a discussion and debate in the art world, as it raised questions about the nature of creativity, authorship, and the role of machines in artistic production. Some critics argued that the painting was simply a product of mathematical algorithms, and that it lacked the intentionality and emotional resonance of human-generated art. One of the most vocal critics of the painting was the artist and critic Jerry Saltz, who wrote a scathing review in New York Magazine in which he called the work "terrible" and accused the art world of "gushing over crap" simply because it was created by a machine.

Other critics raised similar concerns about the role of machines in art, arguing that the painting lacked the emotional resonance and intentionality of human-generated works. Some also questioned the validity of the auction price, suggesting that it was driven more by novelty and hype than by the artistic merit of the painting itself. Others praised the painting as a groundbreaking example of the potential for AI to create new forms of art and challenge traditional notions of authorship and creativity. Despite these debates, "Portrait of Edmond de Belamy" has become a significant cultural artifact and a symbol of the growing interest in AI-generated art. It has also spurred further research and experimentation in the field of computational creativity, as artists and researchers continue to explore the possibilities and limitations of machines as creative tools.

# **4 Conclusion: The Future of Authorship**

The future of authorship is a complex and multifaceted question, as it is tied to larger shifts in technology, culture, and society. However, there are several trends and developments that are likely to shape the future of authorship in the coming years:


Overall, the future of authorship is likely to be shaped by a range of technological, cultural, and social factors, and will continue to evolve in response to changing norms and practices.

# **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Diffusion Probabilistic Model Assisted 3D Form Finding and Design Latent Space Exploration: A Case Study for Taihu Stone Spacial Transformation**

Yubo Liu1(B) , Han Li1, Qiaoming Deng2, and Kai Hu1

<sup>1</sup> State Key Laboratory of Subtropical Building Science, South China University of Technology, Guangzhou, China liuyubo@scut.edu.cn

<sup>2</sup> School of Architecture, South China University of Technology, Guangzhou, China

**Abstract.** Taihu stone is an important landscape element in Chinese Private Garden on Southern Yangtze river, which is known for its profound cultural and aesthetic significance. In this paper, we intend to introduce the 3D spacial character of Taihu stone into architectural forms and spaces using machine learning, trying to explore the possibilities of AI-assisted 3D form finding and design latent space exploration. Existing spacial transformation of Taihu stone is mainly demonstrated by continuous section slicing, which cannot directly analyze and generate 3D space, thus cannot learn the most characteristic internal space of Taihu stone. This paper used the latest 3D point cloud probabilistic diffusion model to achieve 3D form generation and interpolation of Taihu stone and architectural massing through latent space exploration. Experiments show that a sufficiently trained diffusion model can generate 3D point clouds of Taihu stone and building massing, as well as generate interpolations between them. The latent vector can be manipulated to generate outputs that are more oriented towards the Taihu stone or the building massing, to meet the different needs of designers. Generated point clouds can be reconstructed into triangle meshes or voxelized, as a morphological prototype for further design implementation. Generated forms are capable to provide inspiration and reference for the designers to create free forms, showing the potential of the diffusion model to assist architecture design in conceptual phases.

**Keywords:** Deep learning · Diffusion model · 3D form finding · Latent space exploration · Taihu-stone

# **1 Introduction**

The pursuit of innovative forms is a constant topic in architectural design. With the development of deep learning in recent years, form-finding based on the case study and data research brought designers more innovative techniques. Seeking the combination of two elements to develop innovative forms is a common technique in architectural design. For example, exploring the expression of exotic styles in local environments, and discovering new application scenarios for traditional forms and elements in modern contexts. Charles Jencks believes that postmodernism has a dual nature, that is, modern architecture is often combined with something else (usually traditional architectural methods). The solution for the postmodernist movement is to create an architecture that is based on both new technology and old paradigm, as well as being professional and popular [1]. This is usually a challenge for architects. Algorithmic assistance makes design results unpredictable and has great advantages in assisting designers in the early stages of creation and shape finding, but it also requires a new workflow to support this shift in mindset [2]. The machine works in a unique way that can be combined with cultural imagery to produce creative morphologies. A machine can quickly acquire and learn data across historical and geographical dimensions, assisting human architects to understand cultural imagery. In addition, because machine learning is not influenced by emotions or personal preferences, it can also give integrated solutions to different cultural imaginaries and provide a different perspective for human designers, even incorporating different cultural imagery or historical contexts. Campo et al. have combined the characteristics of outstanding examples from architectural history with specific design contexts to generate creative images and inspire the design of 3D architectural spaces [3].

#### **1.1 Traditional Cultural Imagery Transformation in Modern Chinese Architecture**

This paper takes *Modern Chinese Architecture* as an case study. The concept of *Modern Chinese Architecture* is among the most popular topics that Chinese architects have been exploring and practicing in recent years. The core philosophy of *Modern Chinese Architecture* is the inheritance of Chinese elements and the pursuit of innovation. The meanings of *Modern Chinese Architecture* are evolving as time changes, with the connotation of Chinese elements gradually changing from the initial traditional architectural elements such as large roofs to a broader one. Some culturally rich landscapes, components, appliances, etc. have been abstracted and transformed into modern architectural volumes and urban environments.

Architects like Wangshu and Lixinggang are trying to introduce the spatial characteristics of Taihu stone into the architectural space to create innovative forms within the philosophy of *Modern Chinese Architecture*. Taihu stone is an important traditional cultural imagery in the private gardens in China (Fig. 1), with rich cultural connotations (Fig. 2). The morphological characteristics of Taihu stone are exceptionally complex and varied, formed by years of erosion and carving by water or acidic soil in nature, retaining only the hard part of the limestone texture [5]. Its porous and intricate forms coincide with principles of modern architecture in transparency and flowing spaces. For example, in Taihu house designed by Wangshu, the highly abstract geometric shapes of Taihu stone are used as the prototype of the architecture, which reproduced and reinterpreted the spatial characteristics of Taihu stone in the architectural space (Fig. 3).

**Fig. 1.** Taihu stone landscape in Yu Garden, Shanghai [4]

**Fig. 2.** *Dream of the Red Chamber* by Cao Xueqin also named as *The Story of The Stone*. Illustrations in the book reflecting the daily life of the ancient Chinese contain images of Taihu stones. [4]

#### **1.2 Diffusion Probabilistic Models**

The cutting-edge diffusion probabilistic model is introduced into the architectural spatial translation of Taihu stone in this article. The diffusion probabilistic model is a machine learning model inspired by the simulation of the reversed diffusion process developed by Ho et al. in 2015, which can generate high-resolution pictures from random noise [7]. A standard diffusion model is composed of two main process domains: forward diffusion and reverse diffusion. During the forward diffusion period, the original dataset is contaminated with gradually introduced noise until the image becomes completely random noise. In the reverse process, the data are recovered from Gaussian noise by gradually removing the predicted noise at each time step using a series of Markov chains. The diffusion model is more diverse in generating results than the generative

**Fig. 3.** Taihu stone form embedding into architecture by Wangshu [6]

model GAN, which has been developed maturely in recent years, and the process of training and generation is extremely tractable and flexible [8].

This paper intends to use the latest 3D point cloud diffusion model to conduct experiments on 3D shape finding and shape interpolation generation for a complex and porous 3D morphology. Taihu stone is taken as the case in this paper. We try to make a practical transformation of the generated morphology to form a 3D shape-finding workflow at the early stage of design.

# **2 Related Works: Form Finding and AI Creativity**

Generating design alternatives through shape interpolations is a common tool to boost AI-assisted form creation. As early as 1988, Hong et al. used the faceted representation method to achieve interpolation between two morphologies, and the generated results can be used for the simulation of biological evolutionary processes, animation, and portrait robots [9]. Metrics such as the connection between cubical contents (V) and surface quadrature (A) and the relationship between V/A are examined. An interactive platform is constructed to achieve a user-friendly evolutionary design workflow for designers [10]. Recently, machine learning tools such as CNN and GAN-assisted design workflows have made great progress in 2D generations. Corresponding 3D generation is inspired and reconstructed on basis of 2D planar. In deep learning, interpolation generation is often combined with the high-dimensional vectors obtained during training to generate images with semantic meanings. Chen et al. investigated the potential of different representation learning-related techniques in the latent space semantic representation in GAN generation models based on data from the SUN database. The experiment achieved the generation of more diverse and tractable design alternatives [11]. Zhang et al. developed a technique to implement the transformation between 2D pixels and 3D voxels of continuous sections from architectural volumes to build a 2D pixel to 3D voxel workflow [12]. With the workflow, the author improved the ability of machine learning to provide designers with several intermediate solutions between two design styles. However, the generation of low to high dimensions is restricted by manipulation techniques and the designer's own thinking stereotypes. The generated shapes will be confined to predefined rules, and the potential of AI in aiding innovative design will be difficult to explore.

The complex morphology of Taihu stone has also been explored by scholars as a prototype for promoting innovation in complex forms. Feng et al. simulated and optimized the curved surface topology of Taihu stone using CFD and BESO, resulting in a complex porous form [13]. Similarly, Ye et al. developed a computational algorithm for generating tafoni (porous rock morphology similar to Taihu stone), using evolutionary algorithms and 2.5D descriptive algorithms [14]. Furthermore, they explored the practical value of such complex porous forms in architectural design. The aforementioned studies are based on rule-based reconstruction of Taihu stone morphology, which is a cumbersome process and may result in limited outcomes. Notably, Liu et al. employed deep learning techniques to extract and grasp the spatial characteristics of Taihu stone based on labeled cases [15], and the 3D morphologies are reconstructed by continuous sections. However, the experiment did not propose specific methods or references for the translation of the morphological elements of Taihu stone into architectural space, and there is also limitation of dimensions.

It is evident from the related studies that most of the current research in the field of computer-aided form finding and creativity is focused on generating images, with limited emphasis on directly generating three-dimensional forms. Additionally, research on the translation of Taihu stone morphology in architectural space has mostly focused on the study of the complex form of Taihu stone itself, without providing abstract methods that can be directly applied to architectural form. Thus, this paper aims to explore the spatial characteristics of Taihu stone from a 3D perspective using the diffusion model latent space, in order to provide alternative solutions for form finding in three-dimensional architectural space.

### **3 Method**

This paper intends to build a 3D form-finding workflow (Fig. 4) based on diffusion probabilistic models for designers to provide them with morphological alternatives between imagery and target massing blocks. The whole workflow is composed of the development of the front-end and back-end. The back-end work mainly includes training database construction, diffusion model training, and interpolation model construction. The back-end development environment and pre-trained models are packaged as the basis for the front-end development. The front end consists mainly of a user-friendly interface incorporating a GUI, and several 3D reconstruction methods that easy for architects to further develop the architectural spaces. The front end enables the latent code generated by the algorithm not only for the design space but also as a modeling tool that supports multiple operations to generate 3D point clouds.

**Fig. 4.** 3D form finding workflow based on diffusion probabilistic models

#### **3.1 The Back-End**

# **Data Preparation**

This paper aims to propose a paradigm for deep learning to help designers in 3D shape finding, and Taihu stone is selected as a case for the rich cultural connotations it contains. The 3D model dataset of Taihu stone in the experiment is constructed based on the natural generation process of Taihu stone (Fig. 5). The matrix of Taihu stone is limestone, and its complex and porous form was formed through thousands of years of weathering and water erosion. Therefore, in this paper, we simulate the process of generation of Taihu rocks with rhino combined with grasshopper: firstly, 50 samples of high-quality Taihu rocks with different morphological characteristics that meet the aesthetics of traditional Chinese literati are collected as references; the contours of the rocks are generated with Voronoi algorithm; we used surface subdivision on the rock contour to simulate the natural weathering process; finally, the porous morphologies formed by water erosion is simulated with ant colony optimization. To generate the transition form between the building massings and the target Taihu stone, 50 randomly generated building massings are added to the training dataset.

Before training, the data is required to be processed into 3D point cloud format first. The triangular mesh model of the Taihu stone and the randomizer block is sampled into a point cloud format consisting of 8192 points and normalized to a spatial coordinate

**Fig. 5.** Data Preparation

system between -2 and 2, which facilitates the extraction of features by the deep learning model.

#### **Model Training**

The model used in this paper is a 3D point cloud diffusion model developed by Luo et al. based on the standard diffusion mode [16]. Luo et al. added a PointNet structured point cloud auto-encoder to the standard model so that the model can generate point cloud data by denoising process.

The training process contains two stages: encoding and decoding. Firstly, the processed 3D point cloud data is input to the PointNet auto-encoder, and the input point cloud data is encoded into a 512-dimensional latent space vector Z. Z is added as a parameter to each step of the noise-adding process in the forward diffusion process, and the target of the reverse diffusion process is to predict the noise in the forward diffusion. During the decoding process, the noise addition of the forward process and the denoising of the reverse process are cycled continuously, until the predicted noise generated by the reverse diffusion process is highly fitted to the real noise. The fully trained decoder generates 3D point cloud data that highly reproduces the features of the target dataset (Fig. 6).

#### **Shape Interpolation Algorithm**

The shape interpolation generation is mainly achieved by manipulating the 512 dimensional latent vector z generated from the input samples. Inputting the two target morphologies into a fully trained PointNet auto-encoder can generate latent codes z1 and z2 corresponding to the two morphologies. The line-space algorithm can derive interpolation between the two latent space vectors. A sufficiently trained decoder can decode the high-dimensional interpolation vectors into a new 3D form to generate a transition form between the two target forms (Fig. 7).

In this paper, the fully trained model can generate intermediate shapes of Taihu stone and building volumes. The experimental results show that the model can generate 3D shapes with a mixture of different categories, rather than simply finding the average value of the coordinates of two target point clouds (Fig. 8). The number of interpolations is an adjustable input parameter of the testing model, which is set to facilitate the users to

**Fig. 6.** 3D point cloud Diffusion probabilistic models training process

**Fig. 7.** Shape interpolation algorithm based on the trained model

adjust the features of the generated morphology closer to Taihu stone or cubic massings in real-time.

#### **3.2 The Front-End**

Leveraging on the previous work, the resulting pre-trained model and generation environment can be packaged as the basis for front-end development. The latent space parameters are allowed to be edited by users (Fig. 9), making the latent space of the diffusion model not only an exploratory space that can be used for design research


**Fig. 8.** Results of point cloud generation and interpolation

but also can be transformed into a practical 3D shape-finding and modeling tool for designers. Further reconstruction techniques include the down-sampling of point clouds and reconstructed surfaces via open3D to further optimize morphology and abstracting features for architectural design (Fig. 10).

#### **Mesh Reconstruction**

To enable observation and manipulation by designers, the front end must first reconstruct the 3D point cloud generated by the model into a mesh. This paper uses the surface mesh reconstruction method provided by the open3D platform, and further optimizes the reconstructed mesh for surface refinement in Rhino. Through further subdivision and smoothing of the reconstructed surface, a smooth free-form surface shape is obtained, which can serve as a prototype for free-form architectural forms or skins.

#### **Voxel Down-Sampling**

Another approach is to downsample the generated point cloud, abstracting the main features of the generated shape, and then voxelizing it into a more universal and easyto-operate cuboid form. By adjusting the size of the voxels on the X, Y, and Z axes, the size of the reconstructed voxel units can be controlled. Users can adjust the size and proportion of the voxels according to their requirements for the shape. The generated cuboid form can be edited globally and partially as a prototype for architectural design.

**Fig. 9.** Latent space parameters allow editions

**Fig. 10.** Triangle mesh and voxelization reconstructions of the generated interpolation point clouds

#### **4 Discussions**

This article is based on the cutting-edge 3D point cloud diffusion model and combines various methods for point cloud 3D reconstruction, to establish a designer-friendly 3D shape-finding workflow. The main goal of the workflow is to assist designers in generating alternative design proposals between two target forms. The generated proposals preserve the selected style and site volume while also possessing a certain level of creativity.

The main methodology of this paper is to establish an artificial intelligence-assisted design exploration pipeline based on the diffusion model, to discover the lateral thinking and innovative design capabilities of machine learning based solely on 3D datasets. The pre-trained PointNet auto-encoder in the diffusion model encodes a high-dimensional latent code Z during the generation process. By interpolating Z and inputting it into the pre-trained 3D diffusion decoder, the intermediate shapes between two input forms can be obtained. These intermediate shapes possess elements of both forms and have the potential to further develop into building forms. In the front-end part of this paper, two methods: surface reconstruction and voxelization downsampling are proposed to further abstract the generated forms into building forms, demonstrating the potential of the diffusion model in assisting architects in complex form design.

#### **The Practical Value of Taihu Stone Shape Transformation**

In addition, this paper intends to explore the possibility of artificial intelligence-assisted incorporation of traditional cultural intentions into architectural space and to explore the depth of artificial intelligence can explore in fields of architectural design cognition.

In architectural design research and practice, seeking the combination of two elements to inspire new forms of design is a commonly used approach. The significance of transforming and combining traditional elements such as Taihu stone with modern architecture lies in creating unique architectural forms by reinterpreting traditional elements and integrating them with modern architecture, making buildings more artistic and culturally meaningful. At the same time, the implementation of traditional elements can also raise awareness of historical culture, promoting cultural inheritance and development. In addition, the combination of traditional elements with modern architecture can also expand the ideas and methods of architectural design, bringing more possibilities and innovations to architectural design. Building on the research and practice of predecessors such as Wang Shu and Li Xinggang, this paper studies the transformation and application of Taihu stone formative elements in modern architectural massing, exploring the directions of Taihu stone form translation in practical applications. The form of the "Taihu house" is restored in the reconstruction process, providing a machine-learning interpretation of *modern Chinese architecture*.

**Acknowledgments.** This research is supported by National Natural Science Foundation of China (No. 51978268, No. 51978269).

# **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Text Semantics to Image Generation: A Method of Building Facades Design Base on Stable Diffusion Model**

Haoran Ma1 and Hao Zheng2(B)

<sup>1</sup> School of Design, Jiangnan University, Wuxi, China 6210307146@stu.jiangnan.edu.cn <sup>2</sup> Department of Architecture and Civil Engineering, City University of Hong Kong, HKSAR, China

hazheng@cityu.edu.hk

**Abstract.** Stable Diffusion model has been extensively employed in the study of architectural image generation, but there is still an opportunity to enhance in terms of the controllability of the generated image content. A multi-network combined text-to-building facade image generating method is proposed in this work. We first fine-tuned the Stable Diffusion model on the CMP Facades dataset using the LoRA (Low-Rank Adaptation) approach, then we apply the ControlNet model to further control the output. Finally, we contrasted the facade generating outcomes under various architectural style text contents and control strategies. The results demonstrate that the LoRA training approach significantly decreases the possibility of fine-tuning the Stable Diffusion large model, and the addition of the ControlNet model increases the controllability of the creation of text to building facade images. This provides a foundation for subsequent studies on the generation of architectural images.

**Keywords:** Architecture generation · Stable diffusion · LoRA · ControlNet

# **1 Introduction**

Artificial intelligence has entered a new period of integration as of the twenty-first century. Machine learning, the foundational technology of artificial intelligence, is also the attention of architects. Most research use generative adversarial networks (GAN), which produce building facades (Isola et al. 2016) and layouts (Huang and Zheng 2018), to apply machine learning to generative design. These studies demonstrate that GAN trained on labeled samples is very adept at learning the shape of architectural features and their position arranged in building faces and planes. Supervised GANs like Pix2Pix HD (Park et al. 2019), and cycleGAN (Zhu et al. 2017) require conditional input during both training and inference. However, it is challenging to transition between tasks using this GAN model, which was trained using specific samples. For instance, creating structures with various architectural styles necessitates training several models. Moreover, the sample size is the key impediment. Even while unsupervised models like DCGAN (Radford et al. 2015) can train a lot of samples, handling downstream tasks is still challenging.

Multi-modal task processing has become a hot area of research in recent years, including text-to-image generation, as the drawbacks of training samples have been greatly reduced. The Stable Diffusion model (Rombach et al. 2021) is a model for text-toimage generating tasks and creates detailed images from text descriptions. Midjourney (Borji 2022) and DALLE 2 (Ramesh et al. 2022) are models that are comparable to this one. These techniques for creating images to text have been applied broadly in disciplines like architectural design. In the AI Spring series of courses co-organized by DigitalFUTURES & FIU DDes in 2022, the application of text-to-image advanced technology in the field of architecture will be discussed.

However, large sizable diffusion model have poor adaptation to tasks requiring the creation of building facades, and it is typically challenging to regulate the training and generation results (Ruiz et al. 2022). Hence, this research starts with the Stable Diffusion model, utilizes the LoRA approach to refine the model, trains on the building facade dataset, and then integrates the ControlNet model to control the generated results to accomplish the accuracy and controllability of the generated results. This will provide an easier creative tool for architects to generate a large number of controllable building facade design results by changing a few prompt words.

### **2 Methodology**

#### **2.1 Network Architecture**

Stable Diffusion is a Text-to-Image generation technique based on Latent Diffusion Models (LDMs) (Rombach et al. 2021). It can generate better outcomes for image generation than the GAN model. Such as Unconditional image synthesis, image restoration (Inpainting), Super-resolution, Text-to-image generation, etc., random Gaussian noise can be gradually denoised after training. There are three essential parts to the stable diffusion model: (1) Variational autoencoders (VAE), which include both an encoder and a decoder. The first preserves significant deep picture features and transforms the image into a low-dimensional latent space representation for U-Net. The latter creates images from representations in the latent space. (2) U-Net is a residual module-based encoder and decoder that decodes low-resolution images into high-resolution images after the encoder compresses the images. (3) Text-Encoder, which translates the tagged sequence to a potential text embedding sequence, transforms the input text into a meaning that U-Net can comprehend and uses to direct the model as it denoises the embedding. In order to facilitate model loading and image generation, this paper uses Stable Diffusion Web-UI as the control system (Fig. 1). The Web-UI enables Stable Diffusion to have a more intuitive user interface and integrates Text-to-Images, Super-Resolution and model training function.

#### **2.2 LoRA and ControlNet**

Microsoft researchers have developed a new technology called Low-Rank Adaptation of Big Linguistic Models (LoRA), which is primarily used to address the issue of large

**Fig. 1.** Stable Diffusion Web-UI interface

model fine-tuning (Hu et al. 2021). The entire Stable Diffusion model used to be slow and challenging to adjust. Although techniques for lightweight large-scale model finetuning, like Textual Inversion or Dreambooth, are growing in popularity, the graphics card's computational power is still quite demanding. Because the model weight is not necessary to calculate the gradient, the LoRA method injects the trainable layer instead of the pre-trained model weight in each Transformer block, drastically reducing the number of training parameters. LoRA fine-tuning is quicker and less computationally intensive while maintaining the same level of quality as full-model fine-tuning.

On the other hand, the diffusion model generates text and images in a highly random manner, making it challenging to manage the outcome. Furthermore, it can be challenging to precisely regulate the final generated content given the information provided in the text. The recently released ControlNet model addresses this issue by controlling the picture production outcomes by adding more conditions to the Stable Diffusion model (Zhang and Agrawala 2023). As a result, it is now much easier to regulate the diffusion model's strong randomness generation results. Many control conditions are included in ControlNet, including Canny Edge, and Segmentation Map, etc.

In this study, we use LoRA to optimize the Stable Diffusion big model and train it on the CMP Facades dataset, and then we apply various ControlNet model conditions to regulate the development of building facades.

#### **2.3 Training Process**

In this study, 200 images from the CMP Facades dataset (Tylecek 2012) are initially chosen at random to serve as training samples (LoRA fine-tuning training requires very few samples, and the results are excellent). These 200 images are then resized into 512 × 512 pixel. Next, use the text data from each image as the training set for the trigger words. Stable Diffusion v1-4 (Rombach et al. 2021) was selected as the base model, and it was adjusted on an NVIDIA RTX 2060 with 6GB of memory (Epoch = 1, Batch Size = 20000, Learning Rate = 0.00001), taking more than 2 h to complete. The model eventually produced a size of 144MB (the file has been opened in the Civitai community, https://civitai.com/models/11661/buildingfacade). Second, we utilized the model supplied by (Zhang and Agrawala 2023) for the ControlNet model (Fig. 2).

**Fig. 2.** Experimental Workflow

#### **3 Results**

#### **3.1 Generation with Different Style Semantic Base on LoRA**

As shown in Fig. 3, we enter a prompt to compare and analyze different image sampling methods and sampling steps (*<*lora: buildingface:0.7*>*, Ultra hd! editorial, rick owens model, iso 400, surrealistic, 8k, full frame, landscape, architecture, Italian Style). The prompt's representation of the LoRA model is *<*lora: buildingface:0.7*>*, where 0.7 stands for the model's weight. The CFG Scale was set to 7. The degree of influence the text has on the results generated increases with decreasing CFG Scale value, while unpredictability increases with decreasing CFG Scale value. The number of sampling steps is also set at 2, 4, 6, 8, and 32.

We discovered that while the Euler and LMS approaches produce similar content at each sampling step, each sampling method creates distinct content at various sample steps. Heun method is similar to the PLMS method. Until the content of the fourth step starts to emerge, the noise content in the second sampling step is random. It should be noted that stages 6 through 8 of the PLMS method alter at random. Also, the results are remarkably similar despite the fact that DPM2 and DDIM use distinct sampling methods. The DPM2 sampling method, which has the highest tag use rate at over 80%, serves as the foundation for the follow-up study (Rombach et al., 2021).

Then we tried the generation effect of different style semantics in the fine-tuned Stable Diffusion model, and the parameter settings were the same as those in Sect. 3.1. Figure 4 shows how the created building facade style adapts to the features of the prompt when the architectural style is changed (for example, to Italianate). In the case of the

**Fig. 3.** Generation results of different sampling methods and sampling steps

new Chinese style, the model generates beautiful Chinese stone railings in the base and captures the features of the large eaves of Chinese architecture. In the treatment of the windows and doors, the complicated decorative lines are eliminated and a supporting scene with pine trees is created in the front. Despite their strong similarities, the Italian, French and Rococo styles each have their own distinctive features. The French style facade has a classical form with carvings and lines in the details, while the Italian style facade has a window frame with elaborate carvings. The Rococo facade has elaborate ornamentation. With the absence of intricate carving and multiple layers of decorative lines, the Modern style facade is the most understated.

On the other hand, utilizing a fine-tuned LoRA model based on Stable Diffusion can produce content that is entirely different from the original dataset and offers a wide range of adjusting options. By quickly generating numerous designs in various styles and types for building facades with just text input, this technique to facade design for buildings is more effective.

**Fig. 4.** Generation results of different style semantic

#### **3.2 Generation of Different Control Model Base on ControlNet**

The diffusion model from text to image has a significant degree of unpredictability when no control constraints are added. In this part, we add generative conditions to the diffusion model using the ControlNet model. Several control models are offered by ControlNet. In this article, we primarily employ the Canny Edge, Segment Map, Depth Map, and MLSD models to provide control conditions for the production of building facades, and we compare the generation of different ControlNet weights (0.2, 0.4, 0.6, 0.8, and 1.0) results.

As can be seen in Fig. 4, the reference base image of the ControlNet model is a building facade with 512 x 512 pixels. The prompt input and parameter settings are the same as in Sect. 4.1. The results show that the canny edge model produces the best results when the ControlNet weight is set to its maximum (*W* = 1.0), preserving the facade layout of the reference image while also taking into account the requirement for prompts. The elevation layout of the reference image was less similar to the results generated by the other models in the same conditions. For example, neither the layout pattern of the reference image nor the artistic requirements of the prompt were maintained in the results produced by the depth map model. The degree of similarity between the generated outputs is particularly high when the weights of the ControlNet model are relatively low (*W* = 0.2), even though the two models ultimately produce different results. However, when the weights were close to 0.4, the model outputs showed markedly different results.

In general, (1) The generated results are affected differently by the various ControlNet control models, with the canny edge model producing more results than the segment map and MLSD models. The depth map model's output has a better sense of spatial orientation. (2) Fewer ControlNet weight values produce more varied results under the same conditions. The building structure gets more similar to the reference object as the weight value rises, while the building facade has less detail. Increasing the weight value, in other words, restricts the machine's reasoning.

**Fig. 5.** Generation results of different ControlNet Model and Weight

We also generated various building facade styles using ControlNet's Canny Edge model, comparing the effects of various weight values on the outcomes. As seen in Fig. 5, when the weight value of ControlNet increases, the architectural style gradually unifies and the building facade's elements become more condensed. For instance, in the New Chinese style, when the weight value is more than 0.4, some elements are still present but the huge eaves' characteristics progressively fade. When the weight value is set to 1.0, the large eaves feature nearly completely vanishes, although the upper right corner still contains some content. In general, ControlNet may be used to successfully manage the consistency of the results that are created and the reference images, but more building facade features are sacrificed. The ideal range for ControlNet's weight value is between 0.6 and 0.8 (Fig. 6).

**Fig. 6.** Generation results of different ControlNet Weight and style semantic

#### **3.3 Final Generation Experiments**

We used the best parameters in our migration experiments. As shown in Fig. 7, we tried to get the model to generate a modern building facade style, and the stable diffusion model fine-tuned using LoRA understood exactly what we wanted to get. Not only that, but the building facade remained consistent with the architecture of the reference image under the control of the CotrolNet model, and this process took only 0.2 s. We then added the words "white and chrome" to the prompt and the model outputted a white facade based on the text. By simply adding text, it was possible to quickly obtain a different output. This will provide architects with a more efficient concept output. The results of our experiments have been presented in the Civitai community.<sup>1</sup>

**Fig. 7.** Examples of migration experiments

# **4 Conclusion and Discussion**

This research proposed a method for generating building facades based on the Stable Diffusion model. The LoRA method is used to fine-tuned the huge model, which was trained on 200 building facades. Also, use ControlNet to regulate the generation outcomes during the future generation process. The controllable operational research from text semantics to building facade generation is completed in this work. The findings

<sup>1</sup> https://civitai.com/gallery/133518?modelId=11661&modelVersionId=13784&infinite= false&returnUrl=%2Fmodels%2F11661%2Fbuildingfacade.

demonstrate that: (1) The fine-tuning training of the Stable Diffusion model using the LoRA model reduces the computational power needs of the graphics card and saves a significant amount of time. (2) The Stable Diffusion model that has been fine-tuned using LoRA is very flexible to tasks involving building facades, and the semantic characteristics of various styles can be effectively included into the outcomes produced. (3) ControlNet can be used to effectively control the building facade generation results' consistency with the reference object structure, but too much model weight would reduce diversity of results. Overall, this makes it easier to design building facades, simply by changing the prompt words and adjusting the model weights to obtain a large number of quality results. Future research could combine morphological generative algorithms with AI to produce more accurate and richer results.

This study still has some restrictions, though. The amount of data is insufficient in the first place since training with additional data necessitates more digital memory space. According to further research, training can be done on a cloud computing platform with more powerful processing capacity. Second, the prompt's input can be improved further, providing more details may result in the production of more high-quality building facade content.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Research on Image-to-Image Generation and Optimization Methods Based on Diffusion Model Compared with Traditional Methods: Taking Façade as the Optimization Object**

Zexi Lyu1(B) , Zao Li2, and Zijing Wu3

<sup>1</sup> Hefei University of Technology, Hefei, China 1079036667@qq.com <sup>2</sup> Anhui Jianzhu University, Hefei, China <sup>3</sup> University of Waterloo, Waterloo, ON, Canada z256wu@uwaterloo.ca

**Abstract.** The intersection of technology and culture has become a topic of great interest worldwide, with China's development embracing this integration as an essential direction. One critical area where these two fields converge is in the inheritance, translation, and creative design of cultural heritage. In line with this trend, our study explores the potential of stable diffusion to produce highly detailed and visually stunning building façades. We start by providing an overall survey and algorithm fundamentals of the generative deep learning models used so far, namely, GAN and Diffusion models. Then, we present our methodology for using Diffusion Model to generate architecture façades.We then demonstrate how the fine-tuning is done for Stable Diffusion is done to yield optimal performance and then evaluate four different training methods of SD. We also compare existing GAN based façade generation method with our Diffusion based method. Our results show that our Diffusion-based approach outperforms existing methods in terms of detail and quality, highlighting the potential of stable diffusion in generating visually appealing building façades. This research contributes to the growing body of work on the integration of technology and culture in architecture and offers insights into the potential of stable diffusion for creative design applications.

**Keywords:** Façade · Diffusion · GAN · Image-to-image · Image generation · Fine-tuning

# **1 Introduction**

Rapid urbanization in China has incited a conundrum of architectural style disarray, necessitating urgent preservation of vanishing features. Façade enhancement, a vital aspect of architectural style, demands collecting, organizing, analyzing, evaluating, and redesigning extant styles. Traditionally, this labor-intensive process yielded subjective outcomes. This study focuses on generating building façades via stable diffusion, initially establishing a dataset of neo-Chinese architectural façades based on component types and distribution patterns. Subsequently, this dataset evaluates the performance of four stable diffusion methods for façade images and tests existing labeled façade datasets.

**Related work**. Over the past decade, generative image synthesis has been extensively researched and applied, particularly in architectural design. GANs [1], which have dominated the field, consist of a generator producing data samples and a discriminator identifying samples as real or generated. Both components, typically U-Nets, iteratively improve until the generator successfully deceives the discriminator. The generator initiates with random noise sampled from a distribution (e.g., Gaussian), while the discriminator, trained on ground truth datasets, outputs the probability of a sample's authenticity. The process minimizes the loss function:

$$\min \max V(D, G) = E\_{\times \sim \rho\_{\text{data}}(\mathbf{x})} [\log D(\mathbf{x})] + E\_{\mathbb{S}^\* \sim \rho\_{\mathbb{S}}(\mathbf{\bar{z}})} [\log(1 - D(G(\mathbf{z})))]$$

Original GAN has limited performance on conditional outputs, so Conditional GAN [2] was proposed by computing the *D*(*x|y*) and *G*(*z|y*). Pix2Pix [5] further improved CGAN by improving generator and discriminator with U-Net and PatchGAN as well as optimizing loss-function using *L1 loss* as below.

$$G^\* = \arg\min \max \mathcal{L}\_{cGAN}(G, D) + \mathsf{T}\mathcal{L}\_{L1}(G)$$

Further work on Pix2Pix by Yu et al. [7] in their paper on architectural façade generation suggest that Pix2Pix perform well in façade generation and façade style conversion after 100 epochs of training (Fig. 1).

**Fig. 1.** The diffusion process for an input image. Going from left to right is the forward process where Gaussian noises are added step by step until the image is completely Gaussian. The goal of the model is to learn the function that best approximates the reverse process, going from step t to step 0.

Diffusion model [8] is another family of latent variable model that had been researched extensively for image synthesis purposes. The main idea behind DMs is to construct a Markov Chain that adds random Gaussian noise to sample image gradually until it is no longer visually meaningful and that learns how to reverse this process. The forward process is defined as below:

$$q(\mathbf{x}\_{l}|\mathbf{x}\_{l-1}) = \mathfrak{N}\Big(\mathbf{x}\_{l}; \sqrt{1-\beta\_{l}}\mathbf{x}\_{l-1}, \beta\_{l}\mathbf{I}\Big) \\ q(\mathbf{x}\_{l:T}|\mathbf{x}\_{0}) = \prod\_{t=1}^{T} q(\mathbf{x}\_{t}|\mathbf{x}\_{l-1}).$$

where *t* denotes the timestep of each operation and beta denotes the variance sched-ule or noise schedule such that (variance schedule).

$$\{\beta\_l \in (0,1)\}\_{l=1}^T$$

This is done by finding the estimating *q*(*xt*−1) conditioned on original data, that is, *q*(*xt*−1|*xt*, *x*0). Hence rewriting the conditional probability using Bayes rule gives:

$$\mathcal{Q}(\mathbf{x}\_{l}|\mathbf{x}\_{l},\mathbf{x}\_{0}) \sim G(\boldsymbol{\mu},\boldsymbol{\beta})$$

$$\stackrel{\sim}{\beta\_{l}} = \frac{1-\overline{\alpha}\_{l-1}}{1-\overline{\alpha}\_{l}} \cdot \beta\_{l}$$

$$\stackrel{\sim}{\widetilde{\mu}\_{l}(\mathbf{x}\_{l},\mathbf{x}\_{0})} = \frac{\sqrt{\overline{\alpha}\_{l}}(1-\overline{\alpha}\_{l-1})}{1-\overline{\alpha}\_{l}}\mathbf{x}\_{l} + \frac{\sqrt{\overline{\alpha}\_{l-1}}\beta\_{l}}{1-\overline{\alpha}\_{l}}\mathbf{x}\_{0}$$

where α = 1 − β, a simplification trick used in forward diffusion process that makes *q*(*xt*−1) can be conditioned on *x*<sup>0</sup> alone. This way with the reverse process defined, the loss function could be modeled as following:

$$E\left[-\log \rho\_{\theta}(\mathbf{x}\_{0})\right] \le E\_{q}\left[-\log \frac{\rho\_{\theta}(\mathbf{x}\_{0:T})}{q(\mathbf{x}\_{1:T}|\mathbf{x}\_{0})}\right] = E\_{q}\left[-\log \rho(\mathbf{x}\_{T}) - \sum\_{l\ge 1} \log \frac{\rho\_{\theta}(\mathbf{x}\_{l-1}|\mathbf{x}\_{l})}{q(\mathbf{x}\_{l}|\mathbf{x}\_{l-1})}\right]$$

By optimizing ρθ , the reverse process, the model's loss function can be modeled by taking the negative log-likelihood function to get to the variational lower bound of the loss. Ho et al. in his paper on DDPM [3] further simplified the loss function and improved the training efficiency by ignoring the weights in the original function and keeping the variance fixed while train only the mean of the normal distribution.

Rombach et al. in the paper Latent Diffusion Model [4], which is the model we will be using for this paper, further improved the training efficiency for generating high resolution images by first encoding the input into latent variable using an encoder network and then feed the lower dimension latent variables into a DDPM-like U-Net architecture for image generation.

#### **2 Methodology**

We propose in this paper to use fine-tuned Stable Diffusion, an implementation of Latent Diffusion Model to conduct façade generation and compare the effect of various diffusion model training methods and parameter sets have on the final generated façades. We also compare the quality of generated façades with previous work on generative architectural façade using earlier methods such as cGAN (Figs. 2 and 3).

#### **2.1 Introduction to Diffusion Training Methods**

#### **2.1.1 Textual Inversion**

Textual Inversion is a feature in the Stable Diffusion model, which allows for personalizing the model by training a small part of the neural network with custom images. This way, the model can be guided to generate new images based on the concepts taught through Textual Inversion. The Textual Inversion process involves feeding a set of images into the model, which then outputs a vector that represents a specific concept. This vector can then be used in the text-to-image generation process to generate new images based on the taught concepts.

**Fig. 2.** An illustration of img-to-img generation. To the left is the original architecture image<sup>1</sup> taken at Song Yang country, Zhejiang Province of China, and to the right are four img-to-img images generated with respect to the prompts listed in the middle.

**Fig. 3.** The architecture for training and tuning LDM to perform façade design tasks. Random seed is also included to add more variety in generated contents.

### **2.1.2 Hypernetwork**

Hypernetwork is a novel concept used to fine-tune models without touching any weights. This technology is widely used in style transfer and has better generalization performance compared to textual inversion. In Stable Diffusion refers to an additional layer that is processed after an image has been rendered through the model. It tends to skew all results from the model towards the training data, essentially changing the model.

The learning rate for the Hypernetwork may be different than the learning rate for the embedding, with a lower value for the Hypernetwork (Table 1).

<sup>1</sup> Original architecture images are from CRCV· The second National Architectural Design Competition of Songyang Rural Revitalization.

**Table 1.** Comparison of three experiments on Hypernetwork structure2

For the training set we selected, the learning rate of the third experiment achieved a good effect, about 70% of the performance can be restored. LN makes sense for training to be more stable by preventing overfitting. Enabling Dropout can prevent hypernetwork overfitting. Custom dropout ratio is not currently supported, with a default of 0.3. Although the extended layer structure can obtain good training effect, the pt file with layer structure 1, 2, 1 occupies about 83.8 MB of real-time memory, while the PT file with layer structure 1, 2, 2, 1 occupies about 167 MB.

#### **2.1.3 DreamBooth**

DreamBooth [9] is an innovative tool for refining text-to-image diffusion models, such as Stable Diffusion, enabling subject-driven generation. The fine-tuning process entails retraining the model with minimal subject-specific images and identifiers, resulting in a model adept at discerning subjects, isolating them from existing contexts, and accurately synthesizing them within new desired settings. Described as a photo booth by its Google research team creators, DreamBooth facilitates the customization of personalized diffusion models with limited training data. Utilizing Imagen as its foundation, the model can be exported as.ckpt, easily integrated into various UIs. While heralded as the preeminent image generation model, it demands a mid-tier gaming GPU and restricts simultaneous usage with other models.

#### **2.1.4 LoRA: Low-Rank Adaptation for Fast Diffusion Fine-Tuning**

LoRA [10] is a technique for adapting pre-trained language models to new tasks by freezing the original model's weights and adding trainable rank decomposition matrices to each Transformer layer. This approach significantly reduces storage requirements while maintaining input and output dimensions. Implemented as a Python package called loralib, it integrates with PyTorch models like HuggingFace. LoRA introduces minimal inference latency and capitalizes on the inherent low-rank characteristics of large models

<sup>2</sup> Test code from https://colab.research.google.com/drive/1qzweYEMIFkG6jPa04tD1MhW WOzgSnDvP?usp=sharing.

by adding a bypass matrix, simulating full fine-tuning. This method presents a simple, effective solution for lightweight fine-tuning.

# **3 Experiments**

We conduct three types of experiments. First one is a comparison of diffusion model with GAN, pix2pix in particular; second one is a comparison of different parameter tunings among LDM, including sampling methods, steps, CFG Scales, img2img redraw etc.; the last one is a comparison of different training methods, Textual Inversion, Hypernetwork, DreamBooth and LoRA on our own generated dataset. We aim to find an efficient, high quality parameter and training methods that can fulfill the exact needs of architects.

# **3.1 Comparison of Façades Generated by Pix2Pix and Latent Diffusion Model**

We first compare Conditional GAN Pix2Pix with the LDM model used by Stable Diffusion. Pix2Pix is one of the most used generative GAN models in many different fields and it has yielded decent quality and accuracy in the area of architectural façade design. Qiu et al. experimented with *Pix2Pix* on façade design and trained their network on CMP3 Façade dataset by Tylecek et al. for 100 epochs. We use the same dataset and train our LDM and presents a comparison of generated façades as in Figure. As can be seen in the comparison, LDM can achieve better quality and se-mantic understanding in the generated façades then those of the Pix2Pix models (Figs. 4, 5 and 6).


**Fig. 4.** CMP Façade dataset

<sup>3</sup> Dataset from https://cmp.felk.cvut.cz/~tylecr1/facade/, hereby declare.

**Fig. 5.** Homemade Façade dataset

**Fig. 6.** Comparison of architecture façades generated from img-to-img translation using Pix2Pix from Qiu et al.'s work and Stable Diffusion from our tuning.

Another advantage of LDM over Pix2Pix is that LDM is an unsupervised model that does not require any labeling on data for training. We used only the original images in CMP Façade dataset for training while Pix2Pix network also used the label images to assist in training to yield optimal results.

# **3.2 Comparison of Images Generated by Different Prompts**

Stable Diffusion, a prompt-based text-to-image model, comprises two key components: Contrastive Language-Image Pre-Training (CLIP) [17] and the generative Diffusion Model. CLIP, a multimodal model, is trained on text and image data to generate textual summaries from images. It transforms input text prompts into embeddings fed into the reverse diffusion process, conditioning generation. Prompt words stem from the model's natural language processing (NLP) scheme and tagged words in initial training materials. These prompts directly influence the final image elements, with accuracy being vital for effective AI-generated images. Thus, prompt selection and design require meticulous attention for optimal results (Fig. 7).

**Fig. 7.** Prompt + PS/Inpainting img-to-img loop iteration

The above figure illustrates the iterative process of img-to-img used in this research. The current workflow involves the use of prompt and post-processing techniques, such as Photoshop (PS) or inpainting. Using the figure as an example, the forward prompt used by the author is "(masterpiece), (best quality), ((façade-one style)), three 2000-squarefoot, two-stories small modern houses, ((two layers)), with windows and a stone façade, modern and angular, set in a mountain with forest landscape, Subsurface Scattering, Glass Caustics, Small modern house, photorealistic, highly detailed, real architecture, ((low saturation)), highly detailed, HD, Cinematic". "façade-one style" is the label/trigger word trained by the author's model, and using this label for image generation can achieve desirable results. () adds emphasis to a term, [] decreases emphasis, both by a factor of 1.1. You can either stack ()/[] for increasing/decreasing emphasis or use the new syntax which takes a number directly-it looks like this:

(word: 1.1) = (word)

(word: 1.21) = ((word))

(word: 0.91) = [word]

The negative prompt used by the author is "lowers, text, error, extra digit, low quality, jpeg artifacts, signature, blurry, normal quality, cropped, worst quality".

When keeping the seed (the starting point of the random number generator) unchanged, different image effects can be generated by changing the prompt or modifying the match degree between the prompt and the generated image, as shown in Fig. 8.

**Fig. 8.** Prompt replacement—CFG Scale X-Y graphs

#### **3.3 Comparison of Images Generated by Sampling Method, Sampling Steps, Classifier Free Guidance Scale, Img-to-Img Redraw Amplitude**

The diffusion model generates clear images from noisy counterparts via a forward noiseadding process and a backward denoising process. This sampling method, crucial for image generation, affects denoising, quantization, and operational speed. This study compares popular methods, including Euler a, DDIM, and the DPM series. Non-linear iterative methods like DPM a and Euler a exhibit declining quality beyond a certain iteration count, while linear iterative methods, such as DDIM/Euler, display an opposing tendency, with quality relying on iteration count. However, marginal effects limit significant improvements beyond a certain point (Fig. 9).

As shown in the figure, the image generation performance is better with the *Euler a* sampling method and Sampling Steps between 50 and 60.

The Classifier Free Guidance Scale (CFG Scale) balances sample quality and diversity by jointly training conditional and unconditional diffusion models without using a sampler. Higher prompt relevance yields increased prompt frequency and reduced objectenvironment fusion, while lower relevance allows greater AI creativity and enhanced fusion.

**Fig. 9.** Sampling Steps–Sampling Methods X-Y graphs

When the Denoising strength is less than 0.5, local modifications will be made directly on the original image. When the Denoising strength is greater than 0.6, elements that match the original image will be rarely seen (Fig. 10).

**Fig. 10.** Denoising strength—CFG Scale X-Y graphs

As shown in the figure, the image generation performance is better with the CFG Scale is between 7 and 10, and the Denoising is 0.59.

#### **3.4 Comparison of Images Generated by the Training Methods: Textual Inversion, Hypernetwork, DreamBooth, LoRA**

After the training models are completed, the variables are strictly controlled and the tags of the generated embedding and DB model are tested (Fig. 11).

**Fig. 11.** Training models—Hypernet Strength X-Y graphs

Hypernetworks differ from Textual Inversion as they fine-tune the model, leading to better generalization and better training aesthetics. DreamBooth can generate good results with just a few input images of a specific object and its corresponding class name (e.g., dog), along with a unique identifier implanted in different textual descriptions. DB is better than Textual Inversion as it inserts training data into the output, leading to high similarity and great results.

LoRA approximates full fine-tuning expressiveness by setting rank *r* equal to pretrained weight matrices' rank, with increasing trainable parameters. Consequently, LoRA converges to the original model, whereas adapter-based methods converge to an MLP and prefix-based methods to a model restricted by input sequence length (Fig. 12).

**Fig. 12.** LoRA's datasets composition schematic

With the assistance of textual prompts, the training dataset for LoRA can be more guided, resulting in more directed and desirable style transfer outcomes.

LoRA offers a lightweight, efficient alternative to full model fine-tuning of Stable Diffusion, outperforming DreamBooth in speed and adaptability. Low-rank adaptation yields compact results (1-6MB) for easy sharing and compatibility with diffusers and inpainting. In some cases, LoRA surpasses full fine-tuning, with potential for checkpoint merging, recipe creation, and enhanced fine-tuning via CLIP, Unet, and tokens. Offering multi-vector pivotal tuning inversion, LoRA models are smaller than 2GB + DB counterparts, enabling rapid training, art style replication, and DB training with minimal VRAM requirements.

# **3.5 Using Loopback Method to Optimize Images**

**Fig. 13.** Using Loopback method to improve image quality

Loopback is a method by Stable Diffusion to use generated image output, in our case, generated façades, as input for the next round of generation. The process is similar to a cycle of repeating image-to-image translation. We set the iteration steps to 2 steps and Fig. 13 is the yielded result. It can be seen that Loopback can provide better details in generated façades.

### **3.6 Using ControlNet to Guide the Façade Generation Process**

ControlNet is a method proposed by Zhang [17] to control the output of a pretrained Diffusion model to achieve better accuracy. It is achieved by having a locked neural network(the original pretrained model) and trainable copy of the original network at the same time and feed the control conditions(i.e., a edge map or line sketch) to the trainable copy and then connect the copy with the locked model layer-wise.

Best practices for using ControlNet is to convert original image into an edge map. Edges or scratches can effectively control the output into desired results. Some edge detection methods we have tested and resulted decent output includes:


**Fig. 14.** Holistically-Nested Edge Detection in img-to-img

**Fig. 15.** Semantic Segmentation in img-to-img

ControlNet along with edge detection and segmentation techniques enables architects to generate façades designs using a sketch drawing or an existing façade image with better accuracy and better alignment to the user's intentions. Edge detection technology plays a crucial role in controlling the creation of images in the Img-to-Img framework, allowing designers to achieve the desired rendering effects in the generated images, as shown in Fig. 14. The involvement of semantic segmentation allows for more accurate differentiation of the various elements in the original image, facilitating better subsequent translation: architectural elements are replaced with new architectural elements, and so on, resulting in better facade generation and better surrounding environment, as shown in Fig. 15.

To apply lighting to generated images, upload the light source image to the image generation area and place the original image in ControlNet, selecting the Depth model, as shown in Fig. 16. Depth [20], a valuable intermediate representation for actions in physical environments, facilitates realistic rendering in scenes by comparing pixel depth values and preventing distant objects from obscuring closer ones.

Due to the inherent principle of img-to-img, which generates images based on the original image with added Gaussian noise, color block distribution is generally similar, but controlling finer details is challenging. With ControlNet's intervention, the model, initially guided by text generation, can now comprehend information extracted from images. Combined with img-to-img, this yields more desirable control outcomes.

ControlNet also supports the combination of multiple models, enabling multicondition control of images. For example, by setting up two ControlNets, the first one controls building façade contours using HED, while the second one manages background composition through Seg or Depth. Adjusting ControlNet weights, such as prioritizing HED over Depth, ensures accurate façade structure recognition, followed by content and style control through prompt words and style models.

**Fig. 16.** Img-to-img combined with ControlNet--take Depth as an example

# **4 Conclusion and Discussion**

Stable Diffusion outperforms earlier models like Pix2Pix in architecture façade generation, excelling in content quality and training efficiency. By adding a bypass matrix, based on the model's low-rank characteristics, LoRA achieves lightweight fine-tuning effectively.

This method offers potential in architectural style consistency and coherence. Despite some non-functionality, the generated images preserve the original photo's composition and color tone, with the structure well-extracted and translated, resulting in logical façade compositions. Utilizing this method during the sketch stage enables designers to evaluate color, form, and composition across multiple schemes.

However, Stable Diffusion has limitations, including potential inaccuracies in recognizing environmental factors, regulations, and engineering functionality. Thus, human experts should review and refine generated façades for feasibility.

Architectural AI's future is promising, providing assistance and inspiration for façade designs and allowing architects to focus on innovative tasks, elevating productivity. While serving as a valuable tool, it should not replace designers' emotional judgment and final decisions. The technology's success depends on the collaborative synergy between designers and AI tools, capitalizing on each other's strengths and weaknesses (Fig. 17).

Despite personal constraints in data collection and hardware configurations, this study addresses key issues in historical and cultural preservation. It targets challenges like updating historical core buildings, maintaining architectural style and quality, ensuring seamless style transitions in transitional zones, and integrating traditional design elements with modern urban functionality. Additionally, the research leverages digital technologies, including diffusion models, semantic ontology methods, and rough set screening, to develop innovative façade design strategies in preservation areas.

Future research will quantify image data for the training method, enhancing the generation of effective, realistic architectural images. Due to the extensive data required for optimal diffusion model training, subsequent work could explore data collection

**Fig. 17.** Extra effect display

and preprocessing collaborations with academic and commercial institutions, as well as employing automated tools for data identification and refinement. This research holds significant implications for urban design and preservation, with potential applications extending beyond the study's scope.

**Funding.** This research was funded by the National Natural Science Foundation of China, grant number 51978226, and the Anhui Province University Outstanding Scientific Research and Innovation Team, grant number 2022AH010021.

# **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Feedback-Based Design Method for Spatially-Informed and Structurally-Performative Column Placement in Multi-Story Construction**

Ekin Sila Sahin1,3(B) , Daniel Locatelli1, Luis Orozco1,3, Anna Krtschil2,3, Hans Jakob Wagner1,3, Achim Menges1,3, and Jan Knippers2,3

<sup>1</sup> Institute for Computational Design and Construction (ICD), University of Stuttgart, 70174 Stuttgart, Germany

ekin-sila.sahin@icd.uni-stuttgart.de

<sup>2</sup> Institute of Building Structures and Structural Design (ITKE), University of Stuttgart, 70174 Stuttgart, Germany

<sup>3</sup> Cluster of Excellence Integrative Computational Design and Construction for Architecture (IntCDC), University of Stuttgart, 70174 Stuttgart, Germany

**Abstract.** This paper presents a feedback-based computational method for the placement of columns in the early design phase of complex multi-story structures. The method integrates a circle packing algorithm, a spring system, and structural engineering simulations within a single script for the reciprocal and informed arrangement of columns in the space. While allowing the users to have an explorative approach, it empowers diverse potentials in multi-story constructions including additional cantilevering spaces around the boundary, increased spatial qualities with large span possibilities, multidirectional structural arrangements, and multi-purpose use of space. As a result, the developed algorithm allows for flexibility by leveraging the design possibilities of grid-based and irregular column arrangements and promotes the integration of structural and design-related constraints in the spatial organization of various building typologies.

**Keywords:** Computational design · Column placement · Complex networks · Organization of space · Multi-story buildings

# **1 Introduction**

#### **1.1 Integration in Multi-story Construction**

Construction is regarded as a slow-to-change sector since technological advancements often take several decades to be significantly implemented (Drewer and Gann 1994; Grübler et al. 1999; Stoneman 2001; Kuklina et al. 2021). The labor-productivity growth of the building industry has been one percent a year over the past two decades, even when other sectors such as manufacturing or agriculture have displayed remarkable development (Barbosa et al. 2017). In essence, construction involves sophisticated,

project-based activities that include interdependent subgroups collaborating on tasks over time (Mahapatra and Gustavsson 2008). As a result of its normative rules, however, the sector becomes more rigid (Geels 2004). Stable perceptions regarding roles and responsibilities lead to predefined boundaries between disciplines. In spite of it being the world's second-largest industry, the low amount of sharing, concealment of knowledge, and lack of integration hinder innovation in construction.

At the same time, urbanization rates have been increasing in all geographic regions over the last seven decades (United Nations: Department of Economic and Social Affairs 2019). The rise is to such an extent that the number of multi-story construction in the last two decades is more than in the previous 115 years (Oldfield et al. 2014). Column-slab systems, in particular, have gained increasing attention due to their impact on material use and longevity of buildings (Hueste et al. 2007; Georgopoulos et al. 2014; Nandy 2016; Meibodi et al. 2018; Santhosh and Kumar 2021; Krtschil et al. 2022). From a design perspective, those systems require several spatial and structural aspects to be considered. This involves the properties of the building materials, the loads to be carried, the arrangement of the linear and surface elements in each story, and how forces are transferred on the structure (Grünbaum 2008). On a global level, it is expected from the design team to balance varying demands including the spatial decisions, the client's interests, the projects' cost, and the overall performance of the proposed design (RIBA 2020). Conventional practices often follow a linear approach despite the need for integrated knowledge. The involvement of sophisticated and standalone software programs favors the gap between disciplines.

#### **1.2 Computational Design for Integration**

Advancements in computational design have formed a novel paradigm in the building industry. Geometry-based tools and their integrated scripting environments have developed new design thinking with generative rule sets, parameters, and logical relationships (Barrios Hernandez 2006; Oxman and Gu 2015). Finite element analysis tools have helped define stresses, deflections, and dynamic behavior even for intricate geometries using sophisticated techniques (Mueller 2014).

Despite the advanced computer technologies, the fundamental concept of the existing processes has remained unchallenged, displaying the computerized version of traditional modes (Menges 2016). Design tools mostly prioritize the generation of articulated geometric shapes regardless of their multifaceted constraints. Similarly, analysis tools mainly analyze predefined geometries and are therefore unsuited for simultaneously informing the design process. In the early design stages of multi-story construction, designers are limited in how to evaluate their design options beyond architectural constraints. Even in the most prestigious architectural projects, engineers end up having subservient positions (MacDonald 2001). The late consideration of structural concerns or the needs of users results in changes that increase the time and cost of the project.

#### **2 Research Aim and Scope**

The need for integrated thinking is particularly apparent in the design of slab layouts in multi-story construction. Slab design requires the consideration of various domains, such as building codes and regulations, load-bearing capacity, the structural performance of the slab, the accessibility of the space, energy performance, serviceability, aesthetics, cost, and material usage, all in a holistic way. Within that context, the placement of columns directly influences the placement of the beams, walls, or other structural elements, the span and structural depth of the building, the spatial organization of the defined spaces and rooms, as well as the arrangement of the circulation elements and service shafts. Hence, the positioning of the columns has a prominent impact on the holistic domains of multi-story construction. This research aims at enhancing informed and creative thinking in early spatial design while ensuring the structural performance of the slab system. It focuses on the development of a feedback-based computational workflow for the placement of columns in the design of complex multi-story structures.

#### **3 Relevant Work**

In the last two decades, computational methods became increasingly popular for designing floor layouts in buildings. Several have focused on determining the structural and architectural schemes for certain layout conditions. For instance, Shaw et al. developed an evolutionary algorithm utilizing the sweep line method to derive column layouts for orthogonal buildings (Shaw et al. 2008). Nimtawat and Nanakorn suggested a coding scheme that identifies beam-slab layouts with rectangular slabs as binary chromosome strings with given column and wall positions (Nimtawat and Nanakorn 2010). Herr and Fischer provided a strategy for the generation of structural column and beam layouts for reinforced concrete structures in China (Herr and Fischer 2013). Muresan et al. optimized the stiffness distribution in a slab layout while preventing the oversizing of elements using a set of floor outlines and column layouts (Mure¸san et al. 2018). By dividing the rooms repeatedly, Mondal proposed an automation process for placing the columns and beams in single-story convex orthogonal floor plans (Mondal 2018, 2021). However, the boundaries or layouts generated in all the above research have been limited to regular and rectangular configurations. Furthermore, they either lacked continuous structural integration or were insufficient to address spatial complexities such as cantilevered spaces. Similarly, other computational workflows have been initiated to self-organize architectural elements of a structure (Alvarez et al. 2019; Schwinn and Menges 2015). Considering the early design phase of multi-story construction, Orozco et al. developed methods for arranging the panel segmentation and the reinforcement of timber slab structures (Orozco et al. 2021, 2022; Krtschil et al. 2022). Nevertheless, these methods excluded the arrangement of columns.

Some research has also highlighted the importance of automated column placement for less traditional configurations in the early design phase and has been influential in the development of this research. Scheuer used agent-based modeling (ABM) to define the configuration of arbitrarily positioned columns in a large concrete structure. However, because of the dynamic shrinking and growing behavior, the system tended to be heavier and less prone to change. Besides, the involvement of many interdependent parameters made the decision-making process more sensitive (Scheurer 2005). Questioning the linearity of the structural elements, Vierlinger et al. included inclined columns in the system and established a symbiosis of functional and architectural variables. Nevertheless, the involvement of thousands of elements and their elimination required some other optimization steps as well as integrated post-processing (Vierlinger et al. 2013). Preisinger used multi-objective optimization to place inclined columns under a roof while avoiding some predefined volumes. In this example, however, instead of approaching the problem with a more organized and informed methodology, a level of randomness has been involved in the process (Preisinger 2022).

# **4 Methodology**

This paper presents a feedback-based computational method for the early-stage spatial and structural design of slab layouts, focusing on the column placement of complex multi-story structures. Unlike standalone software packages, it allows users to integrate several constraints and simultaneously observe the implications of their design considerations in regular or irregular layout conditions. The methodology utilizes those constraints as inputs and outputs while benefiting from algorithmic design thinking.

# **4.1 Input Variables**

Column positions are often prescribed by architects with consideration of the outside boundary of the slab, the desired span, and the arrangement of used spaces and access areas. In this relatively heuristic approach, designers are limited in the involvement of structural requirements.

Following that, the developed method takes several variables and considerations as inputs (Fig. 1), such as:


**Fig. 1.** Input variables displayed on a testing setup

• No-column areas: The spaces to have no columns within the column distribution area. These can include the locations around the openings and exit routes or the areas involving elevator shafts, stairwells, or spatial rooms.

Considering the performance-related calculations, structural inputs such as the material definitions, cross sections, height, and load should also be defined. In conclusion, a variety of inputs are proposed to enhance the flexibility of the method. As it is possible to disable some of the inputs, it is also possible to expand the number of inputs.

#### **4.2 Algorithms, Solvers, and Outputs**

According to the given constraints, the solvers of the system distribute the columns while helping reduce the displacement of the slab and allowing for simultaneous checking of the results within a single script (Fig. 2). For each feature, algorithms and solvers from different fields have been involved in the process, and integrated into the graphical algorithm editor named Grasshopper [Grasshopper 1.0.0007] in the same computeraided design application software Rhinoceros3D [Rhino 7.0].

#### **Algorithms**

The circle-packing algorithm applies a mathematical technique for arranging circles within a given space so that they are tangent to each other and the boundaries of the space. In the construction industry, architects and engineers try to find the best column arrangement within a given boundary area. When the centers of the circles are viewed as the central points of the vertical structural elements, circle packing solves a similar column placement problem as the architects and engineers do. Therefore, the inputs and outputs of the circle packing have a strong correlation with the ones of the column placement problem. The boundary of the circle packing area, the number of circles, and the distance between the circles' centers correspond to the column distribution area, the number of columns, and the span of the slab structure, respectively. While the outcome of the circle packing is efficiently distributed circles within a space, the outcome of the developed algorithm treats the central points of the circles as columns. For this method, the algorithm developed by Daniel Piker for Kangaroo Physics has been applied [Kangaroo 2.42]. Concerning the predominantly geometric approach of circle packing, structural considerations have been involved through a spring system and a parametric structural engineering tool.

Spring system models are used in various applications, from physics simulations to robotics, for simulating mechanical systems' dynamic behavior. They can be seen as the simplest finite element method using one-dimensional elements (Kattan 2008). When defined as networks, the model describes a position at each vertex point as well as a spring along the edges between those points with a stiffness and a length. Following this logic, the spring system between the column locations is concurrently generated and checked while the column distribution solver is running. The deformation has been the limiting factor of the slab design in the selected case study. Considering the distance between each column, the algorithm prevents the columns from exceeding the optimal span while helping reduce the displacement of the slab. The push and pull mechanism of the springs also ensures that the distributed columns are kept away from the fixed column locations and from the no-column areas.

#### **Solvers**

Structural calculations and engineering models are performed through an interactive structural design plug-in named Karamba [Karamba3D 2.2.0]. It is preferred based on its simplicity of use for non-experts and the speed with which it produces responses to different design options. Besides, to repeat a sequence of instructions multiple times, a feedback-based solver named Anemone is utilized [Anemone 0.4]. In consideration of the desired outputs, the distribution of the columns and the spring system are looped.

#### **Outputs**

Considering the nature of co-design (Knippers et al. 2021), slab layout selection is often not solely based on one domain's knowledge. Instead, the optimal solution is mostly the compromise of several constraints involved, which are mentioned in Sect. 2 "Research Aim and Scope". For this reason, the overall algorithm is designed to enable the users to check the outcomes continuously. These consist of spatial arrangements, the location of the columns, their corresponding structural simulations, column reaction forces, and the displacement of the slab. Furthermore, additional features are integrated with different slab systems in mind, such as generating a network of beams and eliminating the underutilized beams, with the flexibility to expand or modify those options.

#### **4.3 Testing Setup**

The methodology has been chiefly explored and developed on a main case study as a testing setup. The case study selection has been based on the critical overview of current multi-story buildings in regard to their environmental and design-related consequences. The timber building sector has built increasingly more multi-story structures over the last two decades (Svatoš-Ražnjevi´c et al. 2022). The carbon sinkage potential (Churkina et al. 2020), low climatic impact (Agustí-Juan and Habert 2017), high strength-to-weight ratio (Ramage et al. 2017), and ease of machinability of timber (Wagner et al. 2020)

**Fig. 2.** Methodology chart explaining the input variables, algorithms, solvers, and outputs.

are among the reasons of the resurgence of the sector. In addition, several studies have discoursed the interrelationship between the use of timber in the construction industry and increased productivity of labor (Mahapatra and Gustavsson 2008; Barbosa et al. 2017; Salvadori 2021). However, multi-story timber construction has still been limited in its architectural vocabulary and spatial design on account of its restricted span range and unidirectional floor plans.

In order to challenge these architectural limitations, the method has been demonstrated on an irregular multi-story timber structure consisting of a curvilinear slab boundary with the potential for varying span and cantilever conditions (Orozco et al. 2021, 2022). The boundary has been designed with the largest width of 16 m, the largest corner radius of 2 m, and the longest cantilevering balcony condition of around 2.5 m. The outline of the testing setup is visible in Figs. 1 and 2. Even though a pavilion-scale timber building has been chosen as a case study, the proposed method can be applied to other available systems with columns such as concrete, steel, or hybrid material systems. Consequently, its implementations to other timber building layouts and material systems are also showcased.

#### **5 Results**

The initial developments of the algorithm were performed on the testing setup. Six columns were distributed on a total area of 112.3 m2. The column distribution boundary was intentionally designed to be challenging for the structural performance of the slab system. The algorithm was also provided with a fixed column, a fixed wall, and a nocolumn area. The cross sections of the slab, beam, column, and wall elements were defined as 30 cm, 15 × 20 cm, 28 cm, and 25 cm, respectively, and the material was set as timber. Constant gravity and slab loads were applied to the structure and the simulations were displayed accordingly (Fig. 3).

**Fig. 3.** Results of the column placement algorithm on a testing setup. **a** Inputs of boundaries, the number of columns, fixed wall, and no-column area are given to the system. **b** Given number of columns is distributed from the center of the column distribution area. **c** Circle packing algorithm continuously placed the columns while avoiding no-column areas. **d** Spring system kept the span in an optimal position. **e** From all the generated solutions, the output of the algorithms is compared, and the desired one is selected considering the maximum displacement, largest beam length, and the reaction forces on the columns.

The columns were successfully placed on the testing setup by the algorithm. Live outcomes of each step were monitored on the same platform while the solvers were distributing the columns and trying to achieve the optimum span range. This included the maximum displacement of the slab, the length of the largest beam, the reaction forces of each column, and the columns' avoidance of the no-column area, fixed wall, and fixed column. The final slab displacement was 0.39 cm with an achieved span of 7.73 m.

To represent its flexibility, the developed algorithm was applied to several other slab boundaries with fewer inputs and different numbers of columns (Fig. 4).

In addition to the multi-directional layout arrangements, it was also possible to get more regular column placements (Fig. 4b). In conclusion, regardless of the shape of the boundary, the desired output conditions could be achieved.

Lastly, the algorithm was applied to real building layouts with several constraints and different materials. In reference to a timber building, the Tamedia Office Building

**Fig. 4.** Versatility of the algorithm on various boundary conditions. **a** Column placement is run on a convex boundary as a heptagon. **b** Rectangular boundary displayed the performance of the developed tool on regular layouts. **c** Concave boundary is tested for irregular arrangements.

in Zurich was selected as a base (Shigeru Ban Architects 2013). Certain details about the building were obtained from the Multi-story Timber Buildings Database (Svatoš-Ražnjevi´c and Menges 2022). As a hybrid material system, 23 Dwellings' slab layout by Muoto Architects was tested (Muoto Architects 2015) with a steel structure and a concrete elevator shaft. For both cases, the number of columns was kept the same as in the real building case (Fig. 5).

The generated results had similar layouts to the existing column arrangements. Through the process, the users could also investigate other cross-section and column arrangement possibilities with their corresponding structural simulations and calculations. At the same time, certain defects were also identified by applying the developed algorithm to real building layouts. For instance, in some cases, sharp edges on the column distribution boundary prevented the movement of the columns. Therefore, small fillets on the corners were integrated for smoother circle packing and spring systems. Besides, having a large number of fixed locations and predefined spaces on the same layout sometimes caused blockages in the distribution path of the columns. Although those were temporarily fixed, future work can identify possible software bugs and ensure the robustness of the system by testing it on several more cases. Taking everything into consideration, the algorithm successfully integrated several constraints, helped improve the decision-making process with explorative parameters, and allowed its users to approach early design cases more comprehensively.

#### 60 E. S. Sahin et al.

**Fig. 5.** Applications on the real building cases with different materials and layouts. The method is applied on a multi-story timber precedent named Tamedia Office Building by Shigeru Ban Architects which was designed with an irregular layout with 48 columns. The performance of the method is also tested on a hybrid building precedent with steel and concrete named 23 Dwellings by Muoto Architects. **a** Initial position. **b** Reciprocal distribution of the columns considering the given inputs. **c** Output of the algorithm.

# **6 Discussions and Outlook**

This paper presented a feedback-based computational method for the placement of columns in the early design phase of complex multi-story structures. It integrated several design-related variables such as the boundary condition, the desired number of columns, the column distribution area, the optimal span range, the locations of fixed columns and walls, and the no-column areas as constants. Constrained by those preferences, the developed algorithm allowed its users to check the outcomes of the structural analysis live, while letting the algorithm produce their optimal layout. As a result, it empowered diverse potentials which are infrequently seen in multi-story timber building design, including additional cantilevering spaces around the boundary, increased spatial qualities with the possibility for large spans, multidirectional structural arrangements, and multi-purpose use of space. To highlight its versatility, the developed approach is then elaborated on existing slab layouts with different boundary conditions, changing numbers of columns, and different material systems.

Following the proposed method, promising fields for further research have been identified. Methodologically, the algorithm selection can go beyond circle packing and spring system, perhaps to agent-based modeling or machine learning methods. These could enable the integration of other column behaviors and user interaction while approximating optimal solutions. In addition, the column arrangements of several slabs in different levels can be simulated to better understand the seismic behavior of corresponding layouts. This can leverage the design possibilities even further, such as to systems with atriums. From a technical point of view, the single-script approach might be expanded with the full integration of a feedback-based structural solver, and with multi-objective parameters. Conceptually, the flexibility of the developed method can allow for the implementation of several other design variables of different fields. As an example, lifecycle assessment-related parameters or other performance criteria such as acoustics and vibration can be incorporated.

Overall, elaborating on the placement of columns, this method presents a reciprocal co-design approach to integrate the constraints of different disciplines involved in the early design phase of multi-story structures. It bases itself on the existing research and provides a user-friendly platform in a widely-used computer-aided design software environment. Unlike other examples, the method enables the generation and evaluation of a multitude of design options in a relatively quick, easy, and simultaneous way regardless of the regularity or irregularity of the given boundary conditions. Besides, it provides the corresponding simulations and calculations of the structural system in the same platform. The developed method has the potential to surpass the architectural and structural constraints of slab design, allowing for higher productivity, sharing, and integration in a variety of stages of multi-story construction.

**Acknowledgments.** The research was realized as a part of the M.Sc. Design Studio in the framework of Integrative Technologies and Architectural Design Research M.Sc. Program (ITECH) at the University of Stuttgart, led by the Institute of Computational Design and Construction (ICD) and Institute of Building Structures and Structural Design (ITKE). The authors would like to thank Dr. Felix Amtsberg, Simon Bechert, and Daniel Sonntag for their support in the teaching of this Design Studio. Besides, they acknowledge the support of the German Research Foundation under Germany's Excellence Strategy—EXC 2120/1-390831618.

#### **References**


Georgopoulos, C., Minson, A., Minson, G.: Sustainable Concrete Solutions (2014)

Grübler, A., Naki´cenovi´c, N., Victor, D.G.: Dynamics of energy technologies and global change. Energy Policy **27**(5), 247–280 (1999). https://doi.org/10.1016/S0301-4215(98)00067-6

Grünbaum, C.: Structures of tall buildings (TVBK-5156) (2008)


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **On the Development of Timber Structures Based on 3D Interactive Vector-Based Graphic Statics (VGS)**

Jean-Philippe Jasienski(B) , Denis Zastavni, and Sylvain Rasneur

Faculty of Architecture, Architectural Engineering and Urbanism, SST/LAB—Louvain research Institute for Landscape, Architecture, Built Environment UCLouvain [LOCI], Place du Levant 1 (L5.05.02) - B 1348, Louvain-La-Neuve, Belgium jean-philippe.jasienski@uclouvain.be

**Abstract.** The present contribution addresses the topic of how to design novel structures in timber with the aid of a computational tool based on vector-based graphic statics (VGS) in a research-by-design approach. The context, scope and theoretical framework allowing to design strut-and-tie models in timber is explained. An application (design task given to Eng. Arch. Students) is presented. The results concern the primary structure and joints, and are discussed regarding the initial objectives.

**Keywords:** Timber Construction · Structural Design · Digital fabrication · Parametric Design · Graphic-statics · VGS · Research-by-design · Teaching of structures

# **1 Context and Scope**

**Abstract**. The first section explains how the latest developments of graphic statics combined with the use of timber can help tackling the issue of the design of low embodied carbon load bearing structures in the coming years.

### **1.1 The Design of Structures as a Multi-factorial Problem**

The design of structures is a multi-factorial exercise that requires a permanent anchoring in the physical context of the project. In addition to ensuring the mechanical resistance of the structure, the design needs to consider many different factors such as functionality, geometry, construction, cost, and environmental impact among others. In a world of limited and dwindling resources, the structure should also be designed with a view to disassembly, recycling, and reuse of its components.

A structural solution is only optimal regarding to the order of importance given to each criteria. The first challenge lies in defining this weighting. Architects are used to dealing with project involving a set of complex and different constraints. Structural relevance is often left aside in this case unless it reveals unavoidable in the definition of the project. The methodology presented here aims to give a simultaneous focus on both the structural and architectural requirements.

#### **1.2 Graphic Statics and the Design of Structures**

#### **1.2.1 Development of Graphic Statics**

Regarding the design and analysis of structures, graphic statics has proven its considerable potential for achieving efficient and elegant structures. This method relies on two interdependent diagrams, namely the form and force diagrams (Maxwell 1864). The first represent the geometry of the structure together with its external force, the latter embeds a synthetic vector representation of the forces applied to each node of the structure, thus representing vectorially the equilibrium of forces acting on the structures.

The interdependency of the two diagrams and their visual convenience provides a visual and intuitive understanding of the relationship between a structural shape and its inner stresses. Graphic statics was initially developed by the likes of Stevin (1586), Varignon (1687), Rankine (1858), Maxwell (1864), Culmann (1866), and Cremona (1867) among others. The swiss engineer Robert Maillart is one of the pioneers using graphic statics to define innovative and efficient structures. An iconic example is the Salginatobel Bridge built in 1929 (Fivet and Zastavni 2012) This method relying on hand-drawing was almost abandoned in the second half of the 20th century, in part, due to the development of computers and numerical tools (Fig. 1).

**Fig. 1.** Salginatobel Bridge (1929) from Robert Maillart. Left: picture taken by Zastavni (2008). Right: graphic static drawing: with form diagram on the right.

### **1.2.2 Resurgence of the Interest of Graphic Statics**

The resurgence of interest for graphic statics among engineers, architects and researchers in the last years can somehow be explained as follow.

Numerical analysis approaches are not the best for the early stage of design since their way of working relies more on an analytical process with structural models requiring various hypothesis to give a result, resulting in a lack of interactivity.

Secondly, the possibility to benefit from a computational framework to use graphic statics created a favourable context, since hand drawing can be very time consuming and cannot quickly generate or modify geometries for a project.

Finally, the latest theoretical and practical development of 3D graphic statics opened new possibilities in terms of complex 3-dimensional structural typologies. In this, two main methods were developed, namely the vector-based (D'acunto et al. 2019) and the polyhedral-based (Akbarzadeh 2017; Lee 2018). Both approaches have their own specificities and benefits (Fig. 2).

**Fig. 2.** Graphic statics in 3 dimensions: Form diagram F of a self-stressed tetrahedron (left), polyhedral based force diagram (**b**), vector-based force diagram (**c**). (from D'acunto et al. 2019)

#### **1.2.3 VGS a Computational Tool for Vector Based Graphic Statics**

VGS is a plugin in the digital environment of Grasshopper from McNeel Rhinoceros whose main purpose is to automatically generate 3D vector-based interdependent form and force diagrams. VGS is developed by Pierluigi D'acunto, Jean-Philippe Jasienski, Yuchi Shen and Patrick Ole Ohlbrock. The theoretical foundations are based on vector-based graphic statics (D'acunto et al. 2019) and its implementation results in a computational tool (Jasienski et al. 2023).

The plugin allows the designer to generate 3D structures in equilibrium, as well as to modify them in real-time acting either on the force or the form diagram (while assessing the consequence of the modification on the other diagram). It is thus a very adequate tool for the generation of efficient structures at equilibrium at the very first stages of the design phase (Fig. 3).

#### **1.3 Timber as a Construction Material for the Present and the Future**

The actual environmental crisis asks for a major reduction of our CO2 emissions to limit the effects of global warming. In this context, structural engineers have an important role because the construction industry generates around 40% of gasses inducing GW and 36% waste in Europe. It is important to highlight that the primary structure of buildings represents more than 50% of this impact. In order to follow the goals fixed by the 2015 Paris agreement, the construction industry needs to be carbon-free for 2050. Combined to that, the increase in population calls for building new homes. Both engineers and architects are facing a huge challenge to find new ways of building that have a way better ecological impact.

**Fig. 3.** View of a structural model of a 3D tensegrity structure in the VGS tool within the McNeel Rhinoceros and Grasshopper digital environment. From left to right: Form diagram (**a**), corresponding Force Diagram (**b**), parametric definition with the VGS modules in Grasshopper (**c**)

In this context, timber appears to be a very interesting material because it has the capacity to store CO2 and so is considered as having a positive carbon footprint compared to other materials. The negative GWP from the CO2 point of view of timber calls it to play a major role as a construction material in the coming years. In this perspective, VGS being specifically designed to address structures mainly composed of bars in structural networks fits particularly timber structures that are characterized by such arrangements.

#### **2 Theoretical Framework**

**Abstract**. The second section discusses the theoretical framework in which the research takes places. It introduces the hypothesis behind the use of graphic statics and strutand-tie models for timber, within the framework of the theory of plasticity and more precisely the lower bound theorem.

#### **2.1 Theory of Plasticity**

Until the 20th century, the only approach that was used for the calculation of structure was the theory of elasticity. The concept was firstly introduced by Galilée with the famous example of a cantilevered beam, where he considered the section as uniformly in tension (which is partially incorrect).

At the beginning of the 20th century, the first analysis of results of experiments on steel structures questioned that theory. Gvosdev formalized the first principle of plastic calculation in 1938. His work remained unknown, the first theory of plasticity as we know it today were established by Greenberg and Prager in 1949 under the name of static and kinematic theorems.

The theory of plasticity brings an answer to the inconsistencies of elastic theory, considering the ductility of the material and its consequences on the redistribution of stresses (Baker et al. 1956).

#### **2.2 Lower Bound Theorem**

Both theorem of plastic design were theorized in 1936 by Gvosdev and Feinberg, which Greenberg and Prager proved in 1949. The lower bound theorem, particularly suited for design purposes, says: for an ideal plastic material, any limiting load obtained from a distribution of internal forces is less than or equal to the actual limiting load.

As a result, a graphic statics drawing is one possible solution to the static theorem; equilibrium-based design was born, allowing for the management of different materials, structural typologies, or scales, while respecting the three initial conditions of the plastic static theorem. Robert Maillart's work on detailing his structures can be considered as application of this theorem.

#### **2.3 Plastic Design and STM Approaches**

Thanks to the lower-bound theorem of plasticity, any continuous structural system made of a plastic material can be modeled as a strut-and-tie network. The strut-and-tie modelling (STM) approaches were developed for the analysis of shear-walls and structural details in concrete structures based on plastic theorems.

Modelling the structural behavior of complex structures by strut-and-tie networks is a common practice in structural engineering when Bernoulli hypothesis does not apply, which has been effectively evidenced by several contemporary structural engineers.

Strut-and-tie models are considered by Fivet (2013) as high-level structural abstractions that depict the force path acting inside a structure in the most reduced way. It is composed only of rods in compression—struts—or traction—ties—linking together pin-jointed nodes on which point forces are applied. They can be used as generic abstractions for many types of structures such as pin-jointed frameworks or beams and frames subjected to bending moments, but also to trace lines of thrust in compression-only structures.

#### **2.4 Characterisation and Applicability of STM for Timber**

Plastic principles are used for structural dimensioning of timber in most structural standards. Stress-strain relationship of timber demonstrates clearly plastic capacities in compression, both parallel and perpendicular to fibres. In contrast, timber is fragile against traction forces, particularly perpendicularto fibres, with a resistance below one twentieth of the tensile strength along fibres. In most structures, the required ductility for using the principles of plastic analysis and design is reached through the plastic capacities of timber in compression and the ductility brought by steel components of joints.

Due to limited properties to redistribute force in tension, timber should be given special attention for contact joints alone when modeling it using struts and ties to avoid possible.

In this regard CLT panels, which are composed of several interlocking layers (three, five or seven) of planks placed side by side present some interest. These changes in direction within the material give them stiffness and resistance comparable in both main directions of the plane and enable using the principles of Strut-and-Tie Modelling in the framework of plastic design.

# **3 Research by Design—A Case Study**

**Abstract**. The third section explains how a Research-by-Design approach was proposed to architectural engineering students to address the topic of the design of creative and efficient timber structures. The implementation was focused on boththe primary structure and the joint-systems and fabrication.

### **3.1 Research-By-Design**

An in-depth study integrating the multiple parameters introduced previously (see 1.1) benefits from being based on a research-by-design-type approach, where prototyping can have an important role to play. The final solution is neither necessarily defined nor known. The parametric and adaptive dimension is therefore essential. Typically, research-by-design can be implemented involving master-classes, workshops and project development for architectural contest. This method has the capacity to reveal structural approaches or limitations about complex design issues (Fig. 4).

**Fig. 4.** Research by design main significances [credits Denis Zastavni and Commission RPP, 2021]3.2 Research context—design task

The topic of the design project is an addition to the Queen Elisabeth Music Chapel (located in Waterloo, Brussels), a center of excellence for artistic training with an international dimension and outreach. Its instruction is reserved for highly skilled musicians. At the time of its creation, the famous critic Vuillermoz already described it as a kind of 'modern Villa Medici'.

The students involved in the exercise are asked to design, in the vicinity of the existing buildings, a medium-sized rehearsal and concert hall and three to four pavilions for hosting artists in residence (all in timber structure).

**Fig. 5.** View of the work of group 02: site model and model of the concert hall.

#### **3.2 Research Objectives**

The structure of the main hall must be integrated into the design process by the groups of students from the beginning. Their task is to design, size, and draw their wooden structure: (1) design the structure to support vertical forces—dead loads, live loads -, horizontal forces—wind—in X and Y directions; (2) model the structure on the principle of strut-and-tie modeling; (3) build this model in the Rhino and Grasshopper environment and analyze it with VGS; (4) detail a specific joint of the structure, assuming the sections are cut to transmit maximum forces through direct contact, while avoiding the use of steel fasteners. The joint is designed to make it possible to transfer forces according to their nature (tension/compression): contact planes, possible anchoring of tensile forces, possible assembly of the joint, restoration of continuity of one or more bars, etc. The purpose is to promote the direct transmission of forces between timber elements, in the direction of the fibers. The cuts for the assembly can be made using digital fabrication: cutting or milling.

Students involved in the design exercise have the option to work either with wooden sections or CLT panels, considering the possibility of modeling their structural working with STM.

#### **3.3 Results**

The design and development of the structure progress together through drawings, assessments, and models (see Fig. 5). Various support systems are studied to sustain roofs and facades while withstanding snow and wind loads. The large span is dimensioned based on the necessary capacity and architectural constraints. All or part of the structure is modeled using struts and ties in the parametric environment of Rhino and Grasshopper, allowing for continuous adaptation of the structure.

Here, designers ensure the static equilibrium of their structure by using form-finding tools such as Combinatorial Equilibrium Modeling (CEM) or others approaches. The structure is analyzed using VGS, with its transformation module to refine the structure according to the goals defined by groups of two, such as minimal efforts, minimal sections, required height, specific acoustic shape, or particular zenithal openings, etc. This process of analysis and optimization converges towards the definition of the shape and efforts of the structure. These assumptions are then used to design a joint to be manufactured using digital fabrication (see Fig. 6).

**Fig. 6.** Synoptic of the final results of group 06—from top left to bottom right: structural model, assembly proposal, form diagram, force diagram

The diversity of structural models reflects the different approaches taken by the students in defining at the same time the architectural and structural project as a whole.

A few groups managed the exercise with three-dimensional strut-and-tie modeling. With the help of CEM (Ohlbrock and D'acunto 2020), they were able to achieve a balanced structure despite the complex geometries (see Fig. 7).

The VGS tool allowed for a comparative analysis of the values of forces in the bars of the STM models. This can be done node by node using the vector-based approach of VGS: "Regarding the strut-and-tie network as a form diagram F, the equilibrium of the inner forces within the structure can then be solved iteratively node-by-node using vector-based 3D graphic statics" (see Fig. 7).

Based on this graphic statics diagrams, timber joints can be designed according to applied forces (see Fig. 8). A critical examination of students' assemblies quickly revealed numerous pitfalls, such as managing different loading cases, inadequate bearing surfaces, insufficiently balanced forces, the use of metal fasteners for ductile failure in tension, excessively weakened sections due to notches or facet orientations in relation to the forces, etc.… revealing the complexity of such an exercise.

Generally, the designed assemblies involve mechanisms that quickly lead to failure. For example, in the case of the support of the beam of group 04 (see Fig. 10): if a moment caused the beam to rotate around its support, the tie and the column in tension could cause brittle failure. Conversely, the tie in tension and compression in the column would pull the tie out of the column and also cause failure.

These examples highlight the benefits of a rigorous methodology to design timberto-timber contact assemblies, with the advantage of using three-dimensional structural models designed and analyzed by the VGS tool.

#### **4 Discussions and Future Work**

The paper proposed a research-by-design workflow for the design of innovative and structurally efficient timber structures and timber-timber joints. The methodology was applied to a case study as a multi-factorial design task for a group of 3rd year Engineer Architect students. Even if they didn't meet all of the initial objectives, the resulting designs nevertheless demonstrate the interest of the proposed workflow. Designing these joints with graphic statics and the VGS tool allows for the optimization of the amount of material needed to manufacture a connection under design-specific conditions.

Because of the difficulty to visually represent these complex connections and the forces involved, a node-by-node approach, with superimposing form and force diagrams, is not always sufficient for a fine understanding of the working of assemblies. In future design session, using a representation of contact surfaces would allow to go beyond intuition for a better control of stress trajectories and failure mechanisms.

Considering robotic manufacturing potential of these connections for further work opens up new geometrical possibilities and allows for the elimination of any constraints that may be associated with traditional manufacturing methods.

The parametric nature of the VGS approach in a multi-factorial design project workflow demonstrated his interest for timber design challenges. Based on an adequate representation of the connections and an iterative use of VGS, the organization of future design session would allow a more precise development of timber-to-timber connections for the development of wooden structures.

**Fig. 7. a** Form diagram (left) and forces diagram (right) of the primary structure. **b** Form diagram (left) and forces diagram (right) of the primary structure—Each line represents a project

**Fig. 8. a** Structural model (left) and timber-to-timber joint (right). **b** Structural model (left) and timber-to-timber joint (right). Each line represents a project.

# **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **An Exploration on the Form Design of Movable Structures Based on Uniform Convex Polyhedral Expansion**

Mengman Liu, Chuhua Ding, and Hui Wang(B)

College of Civil Engineering and Architecture, Zhejiang University, Hangzhou, China wang\_hui@zju.edu.cn

**Abstract.** 5 kinds of regular polyhedra and 13 kinds of semi-regular polyhedra are taken as the main research objects in this paper to explore the form design method of polyhedral expansion through the rotation of polygon. Firstly, the expandable range of uniform convex polyhedra is defined and divided into two types of expansion. Then three solutions are proposed, namely, discarding polygonal faces, constructing rigid-foldable origami mechanisms and constructing scissorlike elements, so that the prior unexpandable uniform convex polyhedron can be expanded. These methods extend the range of expandable uniform convex polyhedron, and can provide new form design ideas for frontier fields such as movable furniture (toys), movable art installations, 3D kinetic facades and space architecture.

**Keyword:** Uniform convex polyhedra · Expansion · Movable structures · Form design

# **1 Introduction**

Uniform convex polyhedra are convex polyhedra in which faces are regular polygons and angles are the same, including regular polyhedra, semi-regular polyhedra and infinite prisms. In geometry, two important classes of convex polyhedra consisting of regular polygonal faces with highly symmetrical geometry are regular polyhedron and semiregular polyhedron [3]. A Regular polyhedron is composed of only one regular polygon, which contains 5 polyhedra, while a semi-regular polyhedron is composed of more than one regular polygon, which contains 13 polyhedra (Fig. 1). The number of valences of each vertex of a regular polyhedron and a semi-regular polyhedron is the same, and the size of vector is the same from the centre of the polyhedron to each vertex [2], so there is equivalence of each vertex.

Polyhedral expansion is a process in which the polyhedral faces are regarded as rigid faces and the vertices of the polyhedral faces are hinged with each other to expand through the rotation of the polyhedral faces. The main research objects are regular polyhedra and semi-regular polyhedra. The easy control of polyhedral expansion degree satisfies users' demand for different heights and widths of furniture, which makes it valuable in the field of movable furniture. While the interest and visual appreciation of polyhedral expansion have explored its application potential in the field of movable art installations. At the same time, the homogeneity of the expansion direction of the polyhedron and its internal space with capacity make it an ideal model in the field of space architecture.

**Fig. 1.** Regular and semi-regular polyhedra

## **2 Previous Study**

Buckminister Fuller [4] discovered that an octahedron could be transformed into a hexoctahedron by hinging and rotating the vertices of triangular faces in the octahedron, and named this mode of motion as "Jitterbug" [4] (Fig. 2). Clinton [2] extended the "Jitterbug" mode of motion to regular and semi-regular polyhedra, and proposed three types of transformation, namely face rotation-translation transformation, element rotationtranslation transformation and vertice rotation-translation transformation [2]. Stuart [6] proposed seven rules of definition for regular and semi-regular polyhedra, and studied the process of partial polyhedral expansion [6]. Rotation-translation transformation refers to the transformation in which each surface rotates about its axis, translates along its axis, and maintains connection with one of its paired vertices; the surfaces enclosing the polyhedron will transform into another polyhedral form [6] (Fig. 3). According to above definition of rotation-translation transformation, the motion trajectories of each face and vertex can be determined. The vertices on the same surface are used to draw circumcircle, and extrude the circumcircle in the direction from the center point to the center point of the surface to get a cylinder. The intersection lines of adjacent cylinders are the motion trajectories of the vertices connecting the adjacent surfaces in the process of motion [5] (Fig. 4).

Dreher [7] proposed the node construction of "constant dihedral hinge" to solve the unstable problem of Jitterbug motion. Verheyen [7] used mathematical tools to describe the motion trajectory of vertices and surfaces in the motion process of Jitterbug, and discussed the expansion of regular and semi-regular polyhedra [7].

**Fig. 2.** Jitterbug mode of motion. *Source of picture* https://wvutoday.wvu.edu/

**Fig. 3.** Rotation-translation transformation *Source of picture* [6]

**Fig. 4.** The movement of the Jitterbug transformation *Source of picture* [5]

# **3 Types of Expansion**

Previous classifications of expansion types are biased towards morphological analysis. In this paper, from the perspective of morphological construction, the expansion of uniform convex polyhedra is divided into the following three types: polyhedral expansion adding hinged points, polyhedral expansion adding hinged links and polyhedral expansion adding hinged faces (Fig. 5).

Polyhedral expansion adding hinged points: Add hinged points at the vertex of the polygonal faces of a uniform convex polyhedron, around which the surface can be rotated to expand or close. Polyhedral expansion adding hinged links: the intersecting edge of the polygonal faces of the uniform convex polyhedron is added with hinged links of equal length to the length of side. The vertices of the polygonal faces are hinged with the endpoint of hinged links, and the polygonal faces are connected with each other through hinged links. Polyhedral expansion adding hinged faces: Add the same polygonal faces as the expanded polyhedron to form a double-layer structure, which is hinged at the center point of the polygon.

**Fig. 5.** Three types of polyhedral expansion

Polyhedral expansion adding hinged points is single-degree-of-freedom, and there are polyhedra cannot be expanded. Polyhedral expansion adding hinged links is multidegree-of-freedom, and all polyhedra can be expanded. Polyhedral expansion adding hinged faces is based on the polyhedral expansion adding hinged points, and whether it can be expanded also depends on the polyhedral expansion adding hinged points. Therefore, the following focuses on the polyhedral expansion adding hinged points, and discusses the expansion of uniform convex polyhedron.

# **4 Study on Uniform Convex Polyhedral Expansion**

#### **4.1 Expandable Uniform Convex Polyhedra**

Among 18 uniform convex polyhedra, 12 can be expanded according to the rotationtranslation transformation mode of motion without increasing or decreasing the faces of the polyhedron. There are gaps in the expansion of a uniform convex polyhedron. According to the number of vertices contained in the closed state of the gap, the 12 polyhedra can be divided into the following two types: The first type, all polygonal faces around a vertex are hinged to expand, and the gap of expansion is rhombus; The second type, all polygonal faces around two adjacent vertices are hinged to each other, with expanded gap of "S" or "Z" shape (Fig. 6).


**Fig. 6.** Two types of 12 expandable polyhedra

#### **4.2 Unexpandable Uniform Convex Polyhedra and Their Solutions**

Among expandable uniform convex polyhedra, there are at most two states of polygon (the state of a polygon refers to the relationship among a polygon and its adjacent polygons). Two states of polygon are similar to the two sides of a parallelogram, and It's not stable. While 6 uniform convex polygons that are not expandable have three states of polygon, which are similar to the relationship among the three sides of a triangle. The three states of polygon define their relative positions, so that they form a stable structure and cannot be expanded (Fig. 7).

In view of this feature, the unexpandable problem can be solved in the following two ways: First, reduce a state of polygon; Second, transform a state of polygon from a non-scalable to scalable structure. The first way can be achieved by discarding polygonal faces, and the second way can be achieved by constructing rigid-foldable origami mechanisms and scissor-like elements.

**Fig. 7.** Unexpandable polyhedra and their state of polygon

#### **4.2.1 Discarding Polygonal Faces**

When the polygonal faces of a certain state are discarded, three states of polygon in the uniform convex polyhedron are reduced to two. And it becomes out of stability, so the original unexpandable polyhedron is transformed into expandable structure. The discarded polygonal faces can be polygonal faces in any state in the polyhedron. Therefore, there are many choices for discarding and various forms of expansion after discarding. Now take Great rhomicuboctahedron as an example. The polyhedron has three states of polygon: State 1, is an octagon (4, 6, 4, 6, 4, 6, 4, 6) surrounded by 4 squares and 4 hexagons; State 2, is a hexagon (4, 8, 4, 8, 4, 8) surrounded by 3 squares and 3 octagons; State 3, is a square (6, 8, 6, 8) surrounded by 2 hexagons and 2 octagons. Therefore, there are three kinds of discarding options. The following figure (Fig. 8) shows three kinds of discarding options of polygonal faces and the state of expansion after discarding.

#### **4.2.2 Constructing Rigid-Foldable Origami Mechanisms**

Rigid origami is an important branch of origami [8]. Therefore, the folding process of the rigid origami pattern is, in fact, the motion of mechanisms. Unlike paper, which can be

**Fig. 8.** Choices for discarding and the state of expansion after discarding

bent and deformed freely, most engineering materials have a certain degree of stiffness, so rigid origami is formed. Rigid-foldable origami mechanisms are rigid foldable plate structures in which rigid plane surfaces are hinged to each other through common edges [1]. The mechanism can be folded into different spatial shapes from flat forms through hinges, and the face only rotates along the hinges without bending and stretching.

The construction of rigid-foldable origami mechanisms to achieve polygonal scaling (Fig. 9):


Figure 10 shows the process of expanding the polyhedron from closed to maximum expanded state with Small rhomicuboctahedron as prototype, using equilateral triangles as scaling polygons to construct a rigid-foldable origami mechanism.

**Fig. 9.** Diagram of rigid-foldable origami mechanism construction

**Fig. 10.** A rigid-foldable origami mechanism of Small rhomicuboctahedron

#### **4.2.3 Constructing Scissor-Like Elements**

A basic scissor-like element is formed by bars that are interconnected along their length by one or more revolute joints—the Intermediate hinges—allowing one free revolution in their (common) plane [1]. Scissor-like elements can be formed by linking scissorlike element together through articulated joints at their end nodes. During the unfolding process of the mechanism composed of scissor-like elements, all the bars move synchronously. Based on variations in the basic scissor-like element—the shape of the bars and placement of the intermediate hinges—three general subgroups can be identified: translational-, polar-, and angulated elements [1]. The Hoberman mechanism is a typical application example of scissor-like elements.

The construction of scissor-like elements to realize polygonal scaling: The edges of the polygonal face selected for scaling are directly replaced by scissor-like elements, which is hinged at the end of the edge to realize the expansion of polyhedron. The following figure shows the process of expanding the polyhedron from closed to maximum expanded state with Small rhomicuboctahedron as prototype, using the equilateral triangles as scaling polygons to construct scissor-like elements (Fig. 11).

**Fig. 11.** A scissor-like elements of small rhomicuboctahedron

#### **4.3 Architectural Applications of Expandable Uniform Convex Polyhedra**

Based on the polyhedral expansion principle which have been discussed, different polyhedra can be transformed to each other by means of face's rotation and translation. If several faces of the polyhedra are able to fold forward, the polyhedra can be folded into plane state. Using rigid origami mechanism to fill the gap of the expansion can realize the transformation from different polyhedra in closed space. Therefore, the installation based on the polyhedral expansion principle can realize different forms of secondary expansion from the plane state.

Taking the transformation between Hexahedron and Hexoctahedron as an example (Fig. 12), the Hexahedron can be expanded from a square plane to a Hexahedron in form, and the Hexahedron can be further expanded to a Hexoctahedron. If the side length of the Hexahedron is 3.6 m and the volume is 46.6 m3, the volume would be 110 m3 after the expansion into the Hexoctahedron, which increased by 2.3 times. Originally onestory space in Hexahedron can be transformed into two-story space after expansion into Hexoctahedron.

**Fig. 12.** Transformation and architectural application between Hexahedron and Hexoctahedron

# **5 Conclusions**

In this paper the expandable range of uniform convex polyhedra is defined, and it is divided into two types of expansion according to the number of vertices in the closed state of the expansion gap. The three methods proposed in this paper achieve the same maximum expansion degree of polyhedra, but each has its own characteristics (Figs. 13 and 14). In addition, there is also a way of expanding by constructing plane hinged tessellations, which needs to be further explored in the future.

The three methods proposed in this paper achieve the same maximum expansion degree of polyhedra, but each has its own characteristics. The method of discarding polygonal faces does not occupy the inner and outer space of polyhedra during their expansion and closure, but the closed state of polyhedra is not completely closed. The method of constructing rigid-foldable origami mechanisms does not occupy space in the closed state, but occupy internal (or external) space selectively in the unfolding process, and the polyhedra are completely closed in the closed state. The method of constructing scissor-like elements occupies both the inner and outer space of polyhedra in the closing and expanding process, and the closing state is not completely closed. These methods extend the expandable range of uniform convex polyhedron, and can provide new form design ideas for frontier fields such as movable furniture (toys), movable art installations, 3D kinetic facades and space architecture.

**Fig. 13.** 6 originally unexpandable polyhedra using 3 ways to expand—1

**Fig. 14.** 6 originally unexpandable polyhedra using 3 ways to expand—2

**Acknowledgements.** This research was supported by the National Natural Science Foundation of China(52378043).

# **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Biomimetic Form-Finding Study of Bone Needle Microstructure Based on Sponge Regeneration Behavior**

Xiaoxu He and Mingyu Sun(B)

School of Architectural and Civil Engineering, Xiamen University, Xiamen, China 1362697595@qq.com

**Abstract.** The concept of "nature-algorithm-structure" refers to a digital design method in architecture that draws inspiration from nature, extracting its mathematical and physical conceptual models to construct structural systems with parameters. This study aims to address the challenge of parametric form-finding in reticular tension structures. By observing the phenomenon of "sponge regeneration", we further illustrate the generation and optimization of reticular tension structures through the hierarchical structures of "monomer"-"path"-"mesh". Tensile structural systems are rebound forms, and their analytical models must account for their nonlinear characteristics and the existence of equilibrium self-course. Starting from the growth dynamics of "sponge regeneration behavior", this paper extracts the logic behind it: sponge monomers combine randomly into partial units under the condition of shredding and discrete, forming a single organism through aggregation. The multi-dimensional bone needle serves as a structural component, enabling multi-axis reorganization, while the multi-directional mesh surface as a morphological component realizes multi-branch reproduction, forming a natural "network tension structure". This study focuses on the biomimetic formfinding of bone needle microstructure, drawing inspiration from sponge regeneration behavior. By analyzing the growth dynamics of sponge regeneration, we aim to develop a better understanding of the principles behind the formation of bone needle microstructure. This finding provides significant reference for the development of modern structures and promotes the bioshape and optimization of tensile structures.

**Keywords:** Sponge regeneration · Microstructure bionics · Reticular tension structure · Parametric form-finding · Finite element analysis

# **1 Introduction**

Sponges are primitive multicellular organisms known for their strong regenerative abilities. Even from scattered fragments or single cells, a sponge organism can divide and regenerate to form a new adult sponge, which continues to grow. Researchers at the University of Ulster in Northern Ireland conducted experiments on the regenerative ability of sponges. They observed that fragmented sponge cells gathered and grew under the action of "aggregation factors". The growth of sponges was monitored at three time nodes—8 min, 704 min, and 1208 min, see Fig. 1. Sponge regeneration and growth behavior. As bone needles supporting the monomer of the structure, the sponge cells formed a tensile overall structure of different shapes through the path of discrete polymerization. This resulted in the arrangement of fine holes for predation and cloaca on the side of the sponge structure [1]. Observing the regeneration behavior of sponges, and studying the basic monomers, polymerization paths, and reticulated epidermis, provides valuable guidance for biomimetic form-finding of building structures.

b) Sponge growth process

**Fig. 1.** Sponge regeneration and growth behavior

The 1957 exhibition of the Federal Garden in Cologne was presented by Frei A concentrated exhibition of prestressed tents designed and built by Otto. In this exhibition, the earliest and most important forms of tensile film were established. This series of membrane structures has the characteristics of light weight, large span, flexible disassembly and installation, etc., which initially reflect the sustainable and vigorous vitality of this building type. Since then, Frey · Otto gained fame and began to have the opportunity to apply flexible membrane structures to public buildings with larger spans and longer lifespans. The modules of the Underwood pavilion evolved from different variants of the 3-pillar tension module, see Fig. 2. Tensile Structure classification. Changing the distance between the upper and lower surfaces of the module and changing the scale between the upper and lower surfaces of the module inform the curvature of the envelope. These changes also create different rotations within each module, causing the envelope to twist in different directions. The structural simulation engine rhinoceros film and kangaroo are essential tools in the process of finding the shape of the pavilion's structure. Compared with other structural systems, tension structures have great advantages. They mainly use tensile members and are lighter and stronger than conventional systems. The tensile structure is gradually extracted from the three elements of "bar", "cable" and "membrane", and Munich Olympic Stadium and BUGA Fibre Pavilion are the typical cases of "cable-membrane" structure; At the same time, the "bar-cable" tension structure came into being, such as Snelson's "Fly" series, Frumar et al.

**Fig. 2.** Tensile Structure classification

Tension, tension integrity, or floating compression is a structural principle based on a system of isolated components pressed within a continuous tension network, arranged so that compression members (usually rods or pillars) do not touch each other and prestressed tensile members (usually cables or tendons) spatially depict the system. Because the members are loaded with pure compression or pure tension, the structure will fail only if the cable yields or the rod bends. This enables each member's material qualities and cross-sectional geometry to be tailored based on the precise load it carries. Because of these patterns, no structural parts experience bending moments, and the system has no shear stresses. This can lead to an exceptional situation.

Architectural biomimetics is a realistic way for guiding human construction operations by the natural world. Sponge growth and regeneration, monomer change, route derivation, and epidermal shape all have significant research implications for the formation of architectural structure. The modern tensile monolithic structure's main body is composed of rod, line, and surface, and its different morphological unfolding has different mechanical properties, and this paper extends the possibility of parametric form-finding of tensile structure by simulating the regeneration mode of sponge.

# **2 Method**

The sponge monomer is discovered to have a fractal phenomenon, the path exhibits a discrete polymerization algorithm, and the environmental performance of the epidermis morphology is simulated to obtain the ideal form-finding condition, see Fig. 3. This study depicts sponge regeneration behavior from the four perspectives of observing the regeneration phenomenon-extracting the logic behind the extraction-constructing the microstructure system-force analysis, and the technical means are based on the Grasshopper platform, using the Rabbit plugin for monomeric fractal construction, the Waso plug-in for discrete aggregation path deduction, and Lunchbox + WeaverBird for mesh epidermal subdivision, see Fig. 4.

**Fig. 3.** Three paths coordinate

#### **2.1 Monolithic Composition**

The sponge is supported by a needle-like "skeleton". The needle keeps the pores open and maintains the shape of a sponge [2], and can be divided into siliceous bone needles and calcium bone needles according to the type of bone needle material, see Table 1. Multiaxial bone needles; According to the number of axes, it can be divided into singleaxis, double-axis, multi-axis bone needles; On that basis, it could be discussed.

The use of a fractal algorithm to expand the basic unit into several dimensions by specifying the initial value, iteration rules, offset angle, and bone needle monomers, see Table 2. Bone needle fractal algorithm. In the middle of the 1970s, a new area of modern mathematics called fractal arithmetic emerged refers to things that are selfsimilar in terms of phenomena, pictures, or physical processes. Mountains and trees

**Fig. 4.** Technical means

**Table 1.** Multiaxial bone needles


exhibit fractal phenomena in nature, and the creation of a mathematical fractal is based on an equation that iterates continuously, or a recursive feedback system. Mandelbro invented fractal geometry in his well-known book "Fractal Geometry of Nature," where fractal algorithms are combined with ideas like chaos and tubers to form complexity science and direct the structure of the mathematical universe.

#### **2.2 Path Derivation**

The aggregation pattern of bone needles was observed under the microscope, and several bone needles were distributed in scattered points [3], and the center point of aggregation was selected and discretely polymerized with adjacent bone needles within the radius. The path generation is analyzed from the three angles of "boundary"—"plane"—"threedimensional", see Table 3. Discrete aggregation. In the boundary magnetic field effect, the magnetic field attraction effect is compiled with green, yellow and red, and the tensile structure monomer is compared with the group formation form, which is divided into four basic units of tension: monomer forward, monomer reverse, group monomer builtin, and group monomer extracorporeal composition. On the basis of this division, two tensile structural forms, built-in "cable-membrane" and externalized "bar-cable", are


**Table 2.** Bone needle fractal algorithm

derived, and under the joint control of tension and tensile force, the monomer follows the discrete polymerization path to generate a group morphology.

#### **2.3 Epidermal Morphology**

Sponge animal skin presents a light lattice, which has high strength, light texture and good tensile performance, selects 150 \* 250 m plane as the basic grid, the vertex of the structure is set at the center point, subdivides and softens it respectively, takes the local climate of Xiamen, China as the site conditions, analyzes the real number of light on the winter solstice, and conducts simulation analysis through Ladybug software, see Fig. 5. Epidermal contrast, and finds that the maximum number of light on the winter solstice can be reached after softening treatment.

# **3 Structural Performance**

#### **3.1 Forward and Reverse Comparison of Monomers**

On the basis of monomer research, the monomer morphology at N = 3 can be divided into two manifestations: built-in and externalized. Regarding the discussion on the builtin monomer, through the discrete aggregation algorithm, the magnetic field boundary conditions are set, the orbiting path is specified, different forms are generated, and the structure is found by digital means, see Table 4. Monomers are contrasted in different directions. Among them, in the composition of the built-in monomer, groups with different morphologies can be generated, and the groups can form the basic skeleton of the tension structure to create a variety of possible forms.

Compared with other traditional building structure systems, the new system with unique charm of the tensile integral structure is still "young", up to now, there is still


**Table 3.** Discrete aggregation

**Fig. 5.** Epidermal contrast


**Table 4.** Monomers are contrasted in different directions

a) Positive and reverse b)Contrast of different heights

no real sense of tensile integral structural engineering works in the world, one of the significant reasons is that the tensile integral structure is a typical flexible structure [4], with the characteristics of large deformation and small strain, and the ability to bear the load is relatively low. The tensile membrane structure allows architects to design a variety of tensile self-balancing, complex and vivid spatial forms, which change with light throughout the day, and the sculptural membrane structure can take on different forms through light and shadow. At sunrise and sunset, light at low incidence angles will highlight the curvature and relief effect of the roof, and when the sun is at apogee, the streamlined boundary of the membrane structure is cast into the ground with a curved shadow, using the light transmission and reflection of the membrane, and the designed artificial light can also make the membrane structure a sculpture of light. The tensile membrane structure is not rigid and deforms under wind or snow loads. The membrane structure adapts to external loads by deformation, during which the radius of curvature of the membrane surface in the direction of the load is reduced until it can resist the load more effectively. The flexibility of the tensile structure allows it to produce large displacement without permanent deformation, the elastic properties and prestress level of the membrane material determine the deformation and reaction of the membrane structure, and the flexible characteristics of adapting to nature can inspire people's architectural design.

#### **3.2 Group Discrete Aggregate Deduction**

Fractal monomers are generated based on the "Rabbit" plug-in in the Grasshopper platform, and then a discrete aggregation algorithm is generated according to the "WASP" plug-in in the platform, see Table 5. Group discrete aggregation. The generation logic of its algorithm consists of four parts: unit-spatial adaptor-component-group. For the tensile membrane structure, take the 3-rod as an example to construct the structure: firstly, the straight rod of the 3-rod is multipiped to make the port, length and thickness of the member have nonlinear morphological changes; Secondly, an adaptor adapted to the space of 3 rods is established to realize the fit between surfaces and surfaces between different spatial adaptors, so that the members can move, rotate, mirror and other operations in the space adaptation. Furthermore, for path derivation, designers can limit the corresponding field conditions (regular, irregular) and path direction (linear, nonlinear); Finally, according to the above conditions, different forms of bone needle morphology are discretely aggregated, and different forms of tensile membrane structures are supported and extended.

#### **Table 5.** Group discrete aggregation

(*continued*)

Similarly, the "bar-cable" structural form is derived on the basis of the "cablemembrane" tension structure. The compression members of the composite tensile integral structure enhances the overall stiffness and stability of the structural system. The stiffness of the structure as a whole is no longer limited to the tension of the prestress, reducing the excess internal stress [5]. Continuous compression members can maintain an effective force flow path and improve structural efficiency. From a mechanical point of view, the continuous compression member reduces the steps of force flow transmission and the force transmission path becomes simpler. The simpler the force transfer path, the greater the structural efficiency. The continuous transmission of the pressure member can maintain the effective transmission of force flow, which greatly improves the structural efficiency. The structural structure is relatively simple, which reduces the difficulty of making stressed members and connecting nodes, and can realize on-site **Table 5.** (*continued*)

b) Group externalization

production and installation, which greatly reduces the size of the prestress, reduces the difficulty of construction, and makes the structural system easier to achieve.

#### **4 Discussion**

The tensile members are used to control the load or stabilize the junction of the whole Structural system, the cable is directly anchored to the rigid skeleton, directly controlling the skeleton, without the need for the action of the brace. Tensile members can be divided into two types according to the force action, one is common to the rigid skeleton system. Support cables for the self-weight (fixed load) of the support structure, and the other type assists the rigid skeleton against additional live loads. Stable cables to avoid deformation of the structural system; According to whether prestress is applied or not, it can also be divided into two types, one is pretension. Cables, the other is non-pre-tensioned cables. The main feature of this structure is the controlled loads, which are mainly used in contact with each other. The compressive members resist external or self-weight loads and are connected to each other by tensile members to enhance stiffness and stability.

Compared with the strut structure, the skeleton structure greatly reduces the amount of prestress to be applied during the construction process. The combination between the compression member and the tensile member is exquisite, making full use of the bearing capacity of the rigid skeleton structure, and the combination is stable [6]. The lightweight cable of the fixed control system enables the lightweight structure to produce high strength. The sponge skeleton further expands the tensile structure. The continuous development of science and technology has prompted the subdivision of specialties, which, admittedly, is conducive to the deepening and deepening of the knowledge system. The development of science and technology will also cause multi-professional barriers and disconnect between them. Old architects, structures.

The building method of masters, sculptors, and craftsmen as "builders" has long been unsuitable for modern society. In the past hundred years, architectural engineering has become the cohesion and synthesis of multidisciplinary majors, and the relationship between architecture and structure [7]. The most close, it requires both division of labor, more detailed research, and coordination and cooperation between them. Structure is the skeleton that supports the building and the material basis for architectural artistic expression, although the structure belongs to science and technology, But there are often artistic considerations involved. Sexual factors. There are many architects, such as Calatrava, who are willing and good at using structural technical means to express architectural art, and have always maintained an emphasis on structural technology in their architectural creation.

# **5 Conclusion**

The overall tensile structure is a stable self-balancing structure composed of compression members and tension members, the compression members are in a discrete state, and the tension members are in a continuous state. The tensile monolithic structure has no stiffness on its own, but is provided by self-stress. Its geometric composition and mechanical properties have not been studied systematically, and by studying the regeneration behavior of spongees, its monomer composition, path derivation, epidermal tissue and tension structure have a high degree of similarity, which provides more biomorphic reference and guiding significance for the development of reticular tension structure. On the one hand, from the built-in perspective, the "cable-membrane" structure in the tension structure is discussed, the influence of the forward and reverse structure of the rod cable on the mechanical properties of the tension membrane is discussed, and then a variety of bone needle evolution structures are obtained by discrete polymerization algorithm, and a variety of tension membrane structures are constructed. On the other hand, the "rod-cable" structure in the tensile structure is disassembled from the perspective of externalization, and the morphological evolution of the overall tensile structure under different compression modes and path conditions is discussed. The combination of the two provides an innovative idea for finding the shape of the tensile structure.

**Acknowledgements.** The work presented in this paper was supported by Natural Science Foundation of China (Serial No.: 51808471) and supported by the Fundamental Research Funds for the Central Universities (Serial No.: 20720220079).

# **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Threading Cellular Architecture Geometries**

Alberto Fernandez González(B) and Nikoletta Karastathi

UCH Faculty of Architecture and Urbanism, UCL The Bartlett School of Architecture, London, England

{alberto.fernandez.11,n.karastathi.17}@ucl.ac.uk

**Abstract.** In the massive computer architecture known as cellular automata (CA), finite-state machines, also known as finite-state automata, are arranged in a discontinuous network that permits local interactions between neighbors. As selforganizing artificial systems, such as neural networks and genetic algorithms, developed from vast systems formed with essential elements and just local interactions, CA is mainly related to artificial intelligence (AI) (a seed interacts with its own neighbors, which are usually just the cells closer to the seed as an activator). In order to produce architectural spaces of various sizes, this research develops digital experiments that analyze the interactions between environmental conditions as input, CA as generator/propagator, and geometrical emergent patterns from knitting and weaving processes as translators/mediators. This method functions as a bottom-up strategy in which information from the environment can influence the activation and deactivation of rules, theoretically fostering a reprogrammable structure that can evolve.

**Keywords:** Cellular Automata · Threading · Knitting · Weaving · Generative Design

### **1 Introduction**

Finite-state machines (FSMs), also known as finite-state automata, are mathematical models commonly used in computer science and engineering. They are used to describe the behavior of a system with a finite number of states, where the current state and a set of inputs can determine the system's behavior.

In the context of cellular automata (CA) [1, 2], finite-state machines are organized in a discontinuous network that allows for local interactions between neighboring elements. This organization is based on the geometric orders that result from crystal growth simulations, which provide a basis for the arrangement of atoms on a nano-metric scale. The previously mentioned discontinuous network allows CA to create complex and diverse behavior patterns. The individual elements can interact and influence each other based on their current state and the inputs they receive from their neighbors. This results in a highly flexible and dynamic system that can adapt and evolve in response to changes in the environment or inputs, contributing to the overall dynamic and adaptive behavior of the cellular automata system.

Finite-state machines are a fundamental component in designing and organizing nano-metric scale systems. They serve as the basis for the arrangement of building blocks in these systems, and their properties play a crucial role in shaping their behavior and functionality. The geometric orders that result from crystal growth simulations provide a roadmap for the arrangement of building blocks, regulating the interactions between each cell (as the minimal unit) and ensuring that they are organized in a way that responds to external (environment) and internal factors (structural). From these principles, in this research, we are exploring the possibilities of CA as a pattern generator and propagator, driven by external and internal factors, using weaving and knitting principles as mediators between the activation and deactivation of cells and possible ways of materialization, using textiles as soft materials in a continuous geometric arrangement (Fig. 1).

**Fig. 1.** Soft Material simulation over a 2.5D as a continuous surface. Image from the Authors.

#### **2 CA from Discrete Points**

Cellular automata (CA) have a strong connection to the field of artificial intelligence (AI) [3]. This is because CA systems can be considered as self-organizing artificial systems that arise from the interactions between simple components without the necessity of human interaction. Self-organizing artificial systems are systems capable of organizing themselves without explicit direction or control from an external source. These systems arise from the interactions between simple components, often referred to as agents, that are capable of communicating with each other and adapting their behavior based on the state of the system. This can result in the emergence of complex behavior and patterns that are not present in the individual components themselves.

Self-organizing artificial systems are often used to model and simulate complex systems, such as social networks, transportation networks, and ecosystems. They are also used in the development of artificial intelligence (AI) and machine learning systems, where they can be used to model neural networks and genetic algorithms [4]. In these systems, the individual components can exchange information and adapt their behavior based on the state of the system. Over time, this leads to the emergence of coordinated and structured behavior, even though there is no central control or explicit direction.

One of the key features of CA is that they are based on local interactions. In these systems, a seed interacts only with its immediate neighbors, which are typically the cells closest to the seed. These local interactions between the seed and its neighbors can lead to the emergence of complex patterns and behaviors. This is why CA are often considered a good model for AI systems, as they allow for creating sophisticated systems from relatively simple building blocks.

In AI, the concept of local interactions is often referred to as "neighborhood interactions". This refers to the idea that a single unit or component in an AI system only interacts with a limited number of other units or components in its immediate vicinity. This concept is key to the design of many AI algorithms, and it has been heavily influenced by the principles and theories developed in the field of cellular automata [5, 6]. This concept is key to the design of many AI algorithms, as it allows for the development of systems that can scale to handle extensive and complex data sets while still maintaining efficiency and performance and producing meaningful results.

In the field of cellular automata, neighborhood interactions are often described within the context of discrete geometric arrays, which represent the discrete positions of the components within the system. These arrays can be defined as regular grids, hexagonal lattices, or other geometric shapes.

By defining the neighborhood of each component, it is possible to specify the range of interactions that occur within the system. For example, a "von Neumann" neighborhood includes the four adjacent components to a central element, while a "Moore" neighborhood includes the eight adjacent components as well as the diagonals [7]. The use of discrete geometric arrays to describe neighborhood interactions in cellular automata is a powerful tool that allows for the simulation and analysis of complex systems. It has been used in a wide range of applications, including modeling biological systems, studying social networks, and designing artificial intelligence algorithms [8].

The use of discrete geometrical arrangements within cellular automata has been an interesting tool for planning in the architectural field [9]. Also, cellular automata models have been utilized for generating and examining intricate spatial patterns in the design field. Through these models, designers can create complicated spatial arrangements by establishing basic rules of discrete geometric configurations [10].

However, CA as an abstract logical machine that uses a binary language in its essence can easily translate its results into the discrete space in a simplified language (usually as voxels), potentially limiting or constraining architectural design by reducing the complexity and richness of the final design. In that sense, it should be noted that CA is only one tool in the design process, and we may incorporate it with other techniques to produce more nuanced and varied spatial results.

Embracing the simplicity and regularity of discrete geometries on one and considering other ways of translating discrete geometries emerges as an opportunity to explore the field of soft materials such as fabrics. As weaving and knitting techniques are based on the principles of discrete geometric arrays and local interactions, it's feasible to translate cellular automata principles and binary results into discrete weaving arrays (Fig. 2) that represent the positions of components within the system.

# **3 To a Three-Dimensional Geometry Beyond Discrete**

Cellular automata operate within three-dimensional space and utilize a generative approach to accumulate the historical process within their discrete structures. This is achieved through local operations that continuously shape and construct emergent forms

**Fig. 2.** Discrete set of 2D points to be translated by a weaving process. Image from the Authors.

over time following predefined rules. This occurs in a highly abstract process (usually binary) and is pre-deterministic if the rules are not in touch with other external or internal elements acting as a "noise" in the generative process [11, 12]. In that framework, to go beyond this representation of three-dimensional discrete geometry as a set of discrete points, lines, and shapes, we will explore ways to create, visualize and materialize relationships between these points. This research develops the idea of making more complex and nuanced representations of space that consider the continuous nature of the physical world (instead of just a discrete approach). This involves incorporating elements from soft materials, such as other forms of continuous material (such as fluids), into the representation of space [13], in addition to traditional discrete elements like points and lines.

Soft materials are materials that have low stiffness and a high degree of deformability. They are characterized by their ability to flow, deform, and adapt to their environment in response to external forces, such as temperature changes, pressure, or electric and magnetic fields. Examples of soft materials include polymers, gels, liquids, and soft tissues such as skin, muscles, and organs [14]. Soft materials are often used in a variety of applications, including biomedical devices, flexible electronics, and soft robotics [15]. One of the key features of soft materials is that they are nonlinear and often exhibit complex mechanical behavior, which makes them challenging to model and understand. As a result, there is a growing field of research focused on understanding and utilizing the unique properties of soft materials for various applications.

In that particular scenario, this research explores the creation of spatial complexities using CA patterns [16] as a reference for threading continuous structures defined by Synchronous and Asynchronous seeds working in discrete environments [17]. As a final step, the results of this generative process are tested as spatial structures, creating soft translations from the CA discrete logic to highly intricate patterns as answers to the information that pushes the system in terms of complexity. The final step in this generative process refers to evaluating and analyzing the patterns that are generated from the cellular automata (CA) system.

Specifically, in this research, we aim to create woven and knitted translations from the discrete logic of the CA system (Fig. 3). In this context, "woven and knitted translations" refers to converting the CA system's discrete logic into highly intricate patterns. These patterns are created by simulating the interactions between simple components in the CA system. They are seen as answers to the information that drives the system in terms of its complexity and density (Fig. 4). The final step of the generative process is crucial, as it provides the researchers with a way to validate and assess the results of the CA system. By testing the spatial properties of the resulting structures, we can determine whether their approach is effective in creating architectural space at different scales and whether it can be used as a tool for designing and building structures in the real world.

**Fig. 3.** Discrete set of 2D points being translated by a weaving recursive process. (Left) and in a linear process (right). Image from the Authors.

**Fig. 4.** Discrete set of 3D points being translated by a weaving recursive process. (Left), in a linear process (center) and in a mixed variation (right) showing different densities and complexities. Image from the Authors.

# **4 Weaving and Knitting as CA Translators**

Weaving and knitting are related techniques for creating textiles and similar structures using yarn or thread. Weaving involves interlacing two sets of threads, the warp and the weft, in a repetitive pattern to create a fabric. The warp threads are arranged lengthwise on a loom and remain stationary, while the weft threads are woven in and out of the warp threads in a specific pattern to create the fabric. On the other hand, knitting involves creating a piece of fabric by looping yarn or thread around a set of needles. The loops are interlocked and held in place by previous rows, creating a three-dimensional material produced from a single thread. Both weaving and knitting involve creating fabric by repeating a pattern of interlocking loops or threads. However, there are some key differences between the two techniques. For example, weaving typically requires a loom, while knitting can be done by hand or with a machine. Additionally, weaving produces a flat fabric, while knitting can create a material with a more complex, threedimensional structure. Weaving and knitting can be seen as Cellular Automata (CA) translators because they are techniques for translating CA's abstract and mathematical principles into physical form. In CA, a discrete system is modelled as a grid of cells, each of which can be in one of several states, and a set of rules determines how the state of each cell evolves based on the states of its neighbors (Fig. 5).

**Fig. 5.** Discrete set of 3D points being translated by a weaving recursive process using a nylon based compound, showing different densities and complexities. Image from the Authors.

Textiles, specifically knitting, provide a material system that can be programmed using computation and code to create complex structures that respond to environmental conditions. By using parametric design and material computation, designers can create a set of rules or instructions that govern the behavior of the knitting pattern. These rules can be adjusted based on the desired properties of the final structure, such as its strength, flexibility, or response to external forces. This computational design process allows for a high degree of control and precision while maintaining the material's flexibility and adaptability. The programmability of knitted textiles is an excellent interface for creating complex emergent forms of variable properties under the application of internal and external forces [18]. Within the past years, there has been ample research on how textiles can be used in an architectural context. Knitting can allow various performances within a continuous single-form system [19]. Such systems with hierarchical relations are great for multi-performative hybrid structures [20]. Cellular automata (CA) patterns can help in developing such programmable patterns for textiles. CA models are computational systems that can generate complex patterns through the repeated application of simple rules to cells in a grid. This approach can generate intricate patterns for knitted textiles that respond to different environmental conditions (these conditions are actually coming from simulated environments using Ladybug as part of the workflow). By incorporating CA patterns into the design process (Fig. 6), designers can create fabrics with emergent properties that respond to specific environmental conditions, such as temperature or humidity. This process allows the creation of optimized textiles for particular environments, resulting in a more efficient and effective use of resources and more functional and comfortable end products.

**Fig. 6.** Discrete set of 3D points (Processing) from solar radiation patterns (using Ladybug) being translated by a weaving process (Grasshopper), in a linear process (left) and in a recursive process (right) showing different densities and complexities. Image from the Authors.

# **5 Knitting Method**

CA patterns have been used so far to create flat 2-dimensional patterns; however, this project explores how the CA logic can be applied and tested in a 2.5 D—3D where the cellular automata logic can be used to examine the translation of CA to three dimensions. This requires a two-step approach, firstly, an in-depth understanding of how computerized knitting machines work and, secondly, how CA patterns can be translated to knitting commands.

**Step 1**: Flat-bed knitting machines, according to Banerjee [21], can have multiple needle beds, with the most common configuration consisting of an array of needles positioned on two beds facing each other in an inverted "V" pattern. These needles work together to make a loop course by pulling a thread or yarn around loops made in a previous pass. The fabric may be manufactured separately or interlinked on the two beds, allowing for the creation of various loop types.

To shape the textile, there are multiple ways to create the pattern and complexity of the shape. This can be achieved through structural variations of the loop. Transfers are one of the key ways of creating and altering the structure. Transfers occur between the two needle beds in double-bed computerized knitting machines to generate specified patterns and designs. Transfers are the process of moving stitches from one needle bed to another and can be done in various ways. One common technique is to use the transfer carriage, which moves stitches from one needle bed to the other while holding them in place.

To explore how Cellular Automata (CA) patterns can be translated into transfers to result in a 2.5D continuous knitting pattern, we followed the steps below:

(a) Generated a CA grid with 30 × 30 cells, with each column representing a needle bed.

(b) Exported PDFs of each of the cross sections.

(c) Translated the CA pattern into a transfer pattern, using black to indicate when to transfer.

**Step 2**: The transfer pattern was then executed on a double bed computerized knitting machine using the following steps:

(d) Transfer the front loops to the rear of the bed.

(e) Transfer the loops from the rear bed to the front.

(f) Knit a full row to secure the loops. This takes place after every transferee.


These steps resulted in a complex 2.5D pattern derived from the CA pattern. The ability to translate the CA pattern into transfer patterns for double-bed computerized knitting machines opens up new possibilities for creating complex and intricate knitted textiles. By exploring the vast potential of CA patterns and how they can be translated into transfers, designers and researchers can push the boundaries of knitted textile design and achieve previously unattainable forms and structures (Fig. 7).

**Fig. 7.** 3D Knitted CA sample (Ecological Wool 2/14 Nm) based on the 30 × 30 from a 2.5D CA system (CA 2.5D Game of Life) being made using a Digital Knitting Machine (Kniterate). Image from the authors.

#### **6 Conclusions**

In the context of this research, the result of the weaving and knitting process can be seen as a physical representation of the emergent patterns that result from applying the CA rules. By testing these physical structures for their load resistance, researchers can gain insight into the complex relationships between the discrete logic of the CA system and the continuous, three-dimensional geometries that result from its translation into physical form.

This approach provides a unique way to explore the relationships between digital algorithms and physical materials, and it has the potential to inform the development of new architectural and engineering solutions that harness the power of both digital and physical systems. In that context, this research project is focused on developing digital experiments that investigate the relationships between environmental conditions, cellular automata (CA), and geometric patterns that emerge from knitting and woven processes. The aim of the research is to use these relationships to create architectural spaces at different scales.

The goal of this research is to demonstrate how digital experiments can be used to explore the relationships between environmental conditions, cellular automata, and geometric patterns and to provide new insights into the process of creating architectural space at different scales that were tested at this stage of development in weaving patterns but the work on knitting is still under development.

By using CA as a translator, the researchers hope to reveal new and innovative ways of designing and constructing architectural spaces that are responsive to environmental conditions and can adapt to changing conditions over time.

In conclusion, the combination of cellular automata and soft materials has the potential to create a novel approach to design aimed at developing dynamic, adaptive, and selforganizing structures that can respond to environmental changes and evolve over time (Using Ladybug as a weather data engine and Galapagos as an evolutionary solver). By leveraging the unique properties of soft materials, such as their flexibility and deformability, and combining them with the principles of cellular automata, a reprogrammable structure can be created that can adapt and change in response to its environment.

This method emphasizes the importance of local interactions and responsiveness to the environment, allowing for a bottom-up design approach (defining the location of seed as i.e.). The activation and deactivation of rules within the system can be controlled by information from the environment, providing a means to reprogram and evolve the structure over time. This idea holds great promise for developing self-organizing systems that can adapt and evolve, with potential applications in various fields such as architecture, engineering, and robotics.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Auxetic Grammars: An Application of Shape Grammar Using Shape Machine to Generate Auxetic Metamaterial Geometries for Fabricating Sustainable Kinetic Panels**

Simin Nasiri(B)

Georgia Institute of Technology, Atlanta, Georgia snasiri6@gatech.edu

**Abstract.** Auxetic materials are materials with a peculiar mechanical behavior compared to other regular materials. Its main difference exists in its reaction to tension. Most materials exhibit a positive Poisson's ratio [1], that is, they laterally shrink when stretched or expand when compressed. On the contrary, auxetic materials exhibit a negative Poisson's ratio (NPR), that is, they laterally expand when stretched or laterally shrink when compressed [2]. In this paper, the significance and role of geometry in auxetic materials' behavior will be investigated. For this purpose, we will be using shape grammar rules with a strong generative tool called *Shape Machine* [3] to create auxetic geometries with their complex behavior out of simple rules. These geometries' applications can be fabricating sustainable kinetic panels for buildings to interact with and adapt to the environment.

**Keywords:** Auxetic material · Geometry · Shape grammar · Shape machine · Sustainable material · Design computation

# **1 Introduction**

In the world of materials, the Poisson's ratio is a dimensionless constant that depends on the direction of an applied load and describes the ratio of negative transverse strain to the longitudinal strain of a body submitted to a tensile load [4]. This name, derived from the Greek word auxetikos, means "that which tends to increase". In theory, for isotropic 3D materials the Poisson's ratio is between −1 and 0.5 While for isotropic 2D materials, it can assume values from −1 to 1 [5] (Fig. 1).

Compared to conventional materials, auxetic materials possess some enhanced properties, such as shear resistance, indentation resistance, synclastic curvature, crash worthiness, and sound absorption [2]. These properties are the reasons behind auxetic material's strange and useful behaviors. For instance:

**Shear resistance**: when the Poisson's ratio decreases, the value of the shear modulus and consequently the shear resistance of the material increases.

**Fig. 1.** Positive and negative Poisson's ratio and its effect on the shape of material [2]

**Fracture resistance**: Materials that possess a negative Poisson's ratio to have better resistance to fracture than "regular" materials. They also have low crack propagation, and more energy is necessary to expand them than in the case of "regular" materials.

**Acoustic absorption**: Auxetic foams due to their texture and geometries have a superior capacity for acoustic absorption than conventional foams.

**Synclastic behavior**: Synclastic behavior is a body's ability to deform in a shape of a dome when it is bent.

**Shape memory auxetics**: Shape memory is the ability of a material subjected to a plastic or semi-plastic deformation to remember and return to its initial shape, size, and form when submitted to a specific thermal stimulation [2].

**Energy absorption properties**: Auxetic materials have better energy absorption properties, too. The cyclic compression tests on auxetic foams showed the damping capacity of auxetic foams was 10 times higher than that of the original foams that were used for making the auxetic foams [6]. Sound absorption and crash worthiness of auxetic materials were also found to be enhanced and improved compared to conventional materials.

**Permeability**: The auxetic materials have superior permeability compared with conventional materials, due to their pore-opening properties [2].

These enhanced properties make the auxetic materials very attractive for many potential applications, such as personnel protection, military use, biomedicine, aerospace, and textiles [7]. This property makes the auxetic materials more sensitive and useful for sensor applications. Protective clothing and equipment are indispensable for some dangerous sports, such as riding, racing, and skating, to protect wearers from injuries by impact forces [8]. In particular, the parts of the body, such as elbows and knees, which are easily injured, need to be protected, so protective pads are usually used in these areas [2].

Another very important application of these materials can be in building systems as a more sustainable material [9]. Similar to the natural systems that adapt to external conditions, kinetic facades can help internal and external environments adapt and therefore improvement of the building's performance by their dynamic adjustments [10] These kinetic systems typically rely on complex expensive mechanisms, like motors and gears. Compliant, kinetic materially flexible, and adjustable, structures present a more sustainable alternative to such systems. Bistable laminates [9] are one instance of these compliant structures which need a little energy for activation and transition between their two phases. Auxetic meta-materials do behave in a manner that can not only suit bistable material, but also kinetic facades in general. Being able to transform between shapes with a negative Poisson ratio and flexible geometry, they can gradually change shape with adjustable and flexible materials that interact with environmental activators like light and temperature. In this paper, we aim to find a way to recognize and generate these geometries for potential applications in material and building sciences (Fig. 2).

**Fig. 2.** Auxetic materials' flexible behavior [1]

# **2 Methodology**

This research aims to find a generative method for auxetic geometries and patterns to create more variations of them in a shorter amount of time and use simple rules rather than complex mathematical algorithms. This means that using shape grammar over other parametric and algorithmic methods has the advantage of first talking with the language of real shapes [11] (visionary) and not programming symbols for those who don't have coding and programming skills and are interested in geometry and its applications. This will make understanding the geometries generation and their patterns much easier and therefore understandable and creative. And second, using the powerful shape grammar tool, Shape Machine enables us for faster generation of hundreds and thousands of variations of each geometrical combination. In the Shape Machine tool, the very core values underlying are the calculations with shapes, the visual treatment of emergence and ambiguity, and the seamless interface in design workflows [3]. Moreover, the conventions of matching under which a shape can be a part of a design is another main feature of shape machine [3]. Its foregrounding of visual rules over symbolic rules (instructions defined in some programming language) provides a robust and disruptive technology which is differentiating this method and makes it a strong generative tool. Having the history of variation also helps us track the auxetic behavior in each stage of the pattern generation. This will have three benefits: 1 and 2 are that each series of tessellation can give us more than one effective auxetic pattern, as it might have auxetic behavior in more than one stage of the generation. For example, it might have auxetic behavior with or without the fourth layer or stage of the geometry, which can be used in different ways. Other than that, one stage of the tessellation might behave as auxetic but not the next, and it will help us to track the geometry and not lose the pattern in its last format. And 3. We can extract the pattern of auxetic behavior in different variations, combine them, and generate new patterns. This tool is a powerful generative tool to not only create new tessellations and patterns but also apply minor and major changes to them and have infinite variations of each with very simple rules that will be explained following.

In this paper, we started with tessellation and Gereh geometries [12] that were comprised of two shapes and already existed in the literature of auxetic behavior.

These geometries simply can provide the quality needed for an auxetic material, which is the negative Poisson ratio. Two shapes are the simplest geometries [13] that are being studied in this research, but there is a wide range of variety in shapes. They can even go beyond the repetitive and periodic patterns up to the aperiodic quasicrystals [14]. But this paper starts with the very basic shapes to establish the study of the mechanisms and then with an extensive analogy applies the type of mechanism to other more complex geometries.

Shape machine works with identity rules, rules that pick up parts without necessarily doing anything to them. The identity rules have identical left-hand sides (LHS) and righthand sides (RHS) and apply under any given transformation grammar to pick up parts in a shape seamlessly reorganizing the underlying structure of the shape [15] "Technically, for shapes *A* and *B*, the shape rule *A* → *A* can apply to a shape *B* whenever there is a transformation *T* that makes the shape *T*(*A*) part of the shape *B*. If the shape *T*(*A*) is part of the shape *B*, the rule subtracts the shape *T*(*A*) from the shape *B* and replaces it with the very same shape *T*(*A*). The resulting shape *B*\_ and the corresponding computation are given below" [3]:

$$\mathbf{B}' = \begin{bmatrix} \mathbf{B} \ -\mathbf{T} \ (\mathbf{A}) \end{bmatrix} + \mathbf{T}(\mathbf{A})$$

Three conditions have structured the outline of the visual queries respectively [3]:


The shape machine in simple words is a machine with a left-hand side and a righthand side. This is basically what is called a shape rule. On the left-hand side, there is the initial existing shape which will be transformed into the one on the right-hand side. There are three main conditions under which the rules will be applied: Isometry, Similarity, and Affinity [16].


3. **Affinity** finds all the above-mentioned versions of the initial left-hand side shape as well as its deformed distorted angels.

As you will see, they become more inclusive in the order of first to last one, and each of them has its own application accordingly.

**Isometry** transformations keep the shape and size of a shape but alter its position and/or handedness. In this kind of transformation, **the LHS shape is detected in the same size, scale, and shape but its Euclidean position can be differentiated, like its direction, rotation, and position** (Fig. 3).

**Fig. 3.** Isometry shape detection in Shape Machine [3]

Visual queries always need not be confined to identical copies. Often designers want to search for similar copies of a shape in smaller or larger versions and in any location and/or possible enantiomorphic or handed versions in a model or series of models. The new unique transformation introduced in the visual query is the scale transformation that changes the size of the shape. Scale transformations combine with Isometric transformations to produce Similarity transformations that keep the shape of a shape but alter its size, position, and/or handedness. The next series of visual queries in Shape Machine are constructed around shapes that consist of straight lines and are searched under **Similarity** transformations (Fig. 4) in a determinate way. In this kind of transformation, the defined shape in the LHS of the rule **can be detected not only in the same size and form but also in different scales as well as different positions and locations**.

**Fig. 4.** Similarity shape detection in Shape Machine [3]

The next step will be to change them to the RHS shape. The third level of transformation is the **Affinity** transformation (Fig. 5) in which **not only the similar to LHS shapes in the design can be detected, but also the shapes that are skewed in scale and even angle also considered**. Under this transformation, it is enough to have the same number of shape edges and vertices regardless of their connection angle or size. This tool is a very strong tool for many purposes.

**Fig. 5.** Affinity shape detection in Shape Machine [3]

**Fig. 6.** Drawscript+ environment—generating the basic tessellation

The process of generating auxetic geometries using Shape Machine started with forming a basic tessellation context. Following the tessellation grammar will be demonstrated. Then for each pattern, the specific rules which result in that pattern will be shown as the Shape Machine program charts. These charts are called Drawscript+. Including the above-mentioned LHS and RHS shapes, the transformation rule, and options to choose the setting of Isometry, Similarity, and Affinity. The machine also enables loops of repetitive rules. For example, Fig. 6 script creates the basic tessellation context for us to facilitate generating some of the desired Islamic patterns. There are three charts starting from 0 and ending to chart number 2. As it is mentioned in the charts the very first rules are JUMP Fig. 3. It creates the loop of start here at chart 0 with no shape, go to the chart number 1, if there is no shape jump to the chart number 2. This is where the first shape is being created as the context tessellation. Mirroring the tessellation and continuing them in rows happens in these loop jumps. After the jumps are finished it will process the rest of the first chart to unify the layers as well as apply the specifying geometry rules (Fig. 7).

120 S. Nasiri

**Fig. 7.** Tessellation generatin history step by step

In Fig. 8, you can see the initial shape by which we started the shape generation process after having our context tessellation. The next rules will be customized for each pattern.

**Fig. 8.** Shape rules for customized pattern—simple cut auxetic

The first and most simple flexible stretchable pattern for an auxetic material comprised of straight cuts [17] (Fig. 9).

The next pattern does emerge from the rotation of a square unit [18]. Knowing the top-down overlook in Islamic tiling, here a bottom-up approach is taken starting from the basic square in the same tessellation context. Rotating squares create a space in between which provides the space for the geometry and material to expand and stretch. Using the rule under different conditions of Isometry, Similarity, and Affinity generated different patterns which are not mentioned here due to the complexity of the geometries. The simplest and most well-known geometry is the simple rotating square [19] which will be investigated in this research.

**Fig. 9.** Rotating square auxetic pattern bottom-up formation

Lim [6] worked on an analytical approach in auxetic geometry. Doing an analogy between the rotating units and other more complex shapes, we can define some rules to predict whether a geometry can act auxetic or not. In this analogy, the ends of the rotating crosses in Fig. 12c are essentially the hinges for the rotating squares. This is what they called type of missing rib model as it is formed from a square grid with alternating vertical and horizontal ribs removed. Arising from this comparison, an analogy can be established on the auxetic mechanism of different auxetic models (Fig. 10).

**Fig. 10.** Rotating square auxetic behavior [5]

**Fig. 11.** Auxetic behavior rules of thumb [5]

Using the rule of thumb to describe the direction of shapes rotation we will have a series of mechanisms. This is illustrated in Fig. 11 for the expansion and contraction of the perforated sheet, rotating square, anti-tetrachiral, missing rib, and re-entrant with models [5]. Tilted-swastika and the tetrachiral models rotate synchronously, i.e., every rotating unit rotates in the one direction during contraction and reverses during expansion. The code +−+− is introduced to refer to the alternating rotational directions. Opposed to this mechanism pattern is where all the units rotate in the same direction, and reverse only when the direction of applied load changes. Hence the code ++++ is proposed for all the chiral models, the missing rib (tilted swastika) model, the perforated sheets.

This analogy can go beyond more complex geometries like the interconnected star group and anti-chiral group. In these groups, the arrays of the geometries also play a role in their flexibility. This array takes place on the exact same crosses we extracted from the rotating squares. Here we tweak them to a more complex starlike shape [20]. For an instance, the square array of interconnected stars provides the context in which the vertices can act as hinges. This happens as an effect of the geometrical direction. If we position the vertices on the direction endpoints it will result in a concave octagon; on the other hand, positioning the center of the edges on the same endpoints for the anti-chiral group will result in a convex octagon in which the vertices again act as hinges, in a different mechanism, but all in auxetic behavior (Fig. 12).

**Fig. 12.** Comparison of different patterns' similar behavior [5]

The interconnected star groups were chosen for research on their behavior. The rules were defined in the Shape Machine to generate interconnected star shapes. Providing a generative tool for 2D geometries, we could manipulate the codes (shapes and rules) and get completely different and sometimes unexpected geometrical results. Each of which could potentially be an auxetic form which we need to evaluate their mechanisms (Fig. 13).

**Fig. 13.** Interconnected star shape auxetic pattern

# **3 Conclusion**

Knowing the different kinds of behavior that auxetic meta-materials demonstrate due to their designed feature, as designers and architects we want to take the most advantage of them for the purpose of more sustainable and adjustable environments. Auxetic materials


**Fig. 14.** Shape Machine generation of the interconnected star

can be applied and used for kinetic panels in different formats and textures. The focus of this paper was on finding the significance of their geometries in a comparative analysis to find out the grammars or rules to be used simply by applying them to the shapes without many mathematical algorithms to solve and generate them. Using Shape Machine for the first time to generate auxetic geometries, we found it a powerful generative tool for designers to be able to create auxetic patterns for their purposes by basic initial shapes and some rules of thumb to compare and analyze their behavior, as well as fast and simple transformations which generate numerous variations. The variety of transformations that Shape Machine provides including Isometry, Similarity, and Affinity provides us with a powerful tool to first detect the shapes and then transform them to the shapes that an auxetic geometry needs to perform. Having the history of each step of transformation also gives us more options both for generating new patterns and analyzing the patterns. These geometries ad their useful behaviors then can be used in multiple applications for a sustainable environment (Fig. 14).

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Graph Constrained Multiple Schemes Generation for Campus Layout**

Yubo Liu1, Zhilan Zhang2(B) , Kai Hu2, and Qiaoming Deng2

<sup>1</sup> State Key Laboratory of Subtropical Building Science, South China University of Technology, Guangzhou, China

liuyubo@scut.edu.cn

<sup>2</sup> School of Architecture, South China University of Technology, Guangzhou, China 376322305@qq.com

**Abstract.** The campus layout is a major stage in the early stages of campus planning and design. When assessing the feasibility of a campus site in stage, we usually compare multiple campus layout schemes, which consumes a lot of time. The design process can be accelerated if multiple campus planning schemes can be generated quickly to meet the desired requirements. This study aims to explore the possibility of using graph neural networks (GNN) to generate multiple campus layouts. We use a step-by-step generation method. The first step is generating campus functional zonings based on user constraints. The second step is generating campus building layouts based on the functional zonings. Ultimately the machine is able to quickly generate multiple campus layout schemes by user input of graph constraints such as the number of functional zonings, the type of functions and their adjacency. In the experiment, we trained 200 campus layout samples and verified the validity and accuracy of the experiment after qualitative and quantitative analysis.

**Keywords:** Campus layout · Graph neural networks (GNN) · Campus function bubble graph · Generative design · User constraints

# **1 Introduction**

As an important technology in the field of artificial intelligence, Deep learning has received a lot of attention since it was introduced by Hinton [2] in 2006. It acquires the complex patterns of large-scale data by mimicking the nervous system in the animal brain to perceive and understand information. Deep neural networks such as generative adversarial networks (GAN) [1, 5] and graph neural networks (GNN) [13] have improved the ability of machines to learn from images, and a number of research have shown that they have certain potential in quickly generating architectural design layouts. However, most of the current studies have focused on the layout of simple apartments, and the results generated lack controllability and diversity.

In the pre-planning stage of campus planning and design, the university administrators need to compare multiple campus layout plans to determine if the site meets the construction requirements. However, they lack mature design and construction experience to complete this step independently. If there is a tool that can automatically generate multiple schemes to assist the them in quickly identifying the direction of campus development, it will shorten the research time for site and facilitate the evaluation work. From this perspective, it is important to explore a method for quickly obtaining multiple campus layout options based on user constraints.

This paper hopes to combine campus layout design research with deep learning technology, and generate a variety of reasonable campus layout schemes that meet user needs through deep learning. The campus construction of universities in China is mainly in three situations: renovation of old campuses, expansion of old campuses, and the build of new campuses, because the renovation and expansion projects of old campuses are relatively complex, this study mainly considers the construction of the new campus. The construction of new campuses is often chosen as a stand-alone site, which usually includes two situations, one is that the campus is built on an urban site with known surrounding site conditions, and the other is the campus is built on a site to be developed with undetermined surrounding conditions.We have explored the former case in previous studies [6, 8, 9], and this study applies to the latter. Using a step-by-step generation method combining graph neural networks (GNN) and generative adversarial networks (GAN), the machine is able to quickly generate multiple layout schemes based on the site contour and the constraint parameters input by the users. Then it can reduce the time spent on pre-design inputs and improve design efficiency.

# **2 Related Work in the Field of Deep Learning Techniques and Buildings Layout Generation**

At present, deep learning technology has penetrated into the field of architectural design. Many scholars have explored the direction of building layout generation, constantly trying out more complex research objects and more advanced technical methods. In terms of the selection of the research object, it has changed from the small-scale layout of floorplan [4] to the medium-scale layout of blocks and residential areas [12], and now to the campus layout and even the urban layout [6–9]. The scale of the object of study is constantly increasing and the elements it contains are becoming more and more complex. The following section will focus on research related to deep learning combined with campus layout generation. Liu et al. [8] used deep learning techniques for primary school and campus layout generation and summarized a labelling method for small sample generation. Lai [6] explored campus layout generation methods using pix2pix model. The study took the central loop campus as the research object and proposed a step-by-step training method. The experimental results demonstrated that the step-bystep generation method can make the machine better learn complex layouts. In addition, the authors' comprehensive summary of strategies to achieve building layout generation based on small samples provides a good inspiration for the research in this paper. However, the current study can only output a single result based on a single site condition. Since then, Liu et al. [9] have explored and experimented with the diversity of output results, proposing an innovative training method. This method allows for the simultaneous input of two images, a functional bubble image of the campus site boundary image, by changing the image input channel of the pix2pix model. The user can change the input campus function bubble diagram to control the generated results. The experiment achieved the goal of generating a diverse campus layout in the same site conditions, but lacked controllability and clarity of the generated results. Our research hopes to continue the exploration of diversity and controlled generation of campus layouts in combination with the experience of previous studies.

In terms of the diversity and controllability of the generated results, current research has been explored using different deep learning algorithms. Many scholars have taken different perspectives to explore the constraints contained in design or to disassemble the design process, and they have focused on how to better learn the design logic involved [17]. Pan et al. [12] used the GauGAN model to generate the neighborhood layout of northern China. This method can generate diverse results by changing the style of the input image. However, the final result changes are very weak and it is difficult to see the direct effect of the input elements on the results. Hu et al. [3] proposed Graph2Plan based on graph neural networks (GNN) and convolutional neural networks (CNN). The idea of the method is that the machine searches for similar layouts from a database based on user input constraints. It then generates a new suitable layout on the given input boundary based on this layout. Nauata et al. [10, 11] propose the House-GAN model to explore the generation of user-constrained floorplan. Different from Graph2Plan, this study combines a graph network relational model with GAN, using a generator-discriminator adversarial learning method. It encodes user constraints such as the number and type of rooms and their spatial adjacencies as graph input the machine. Diverse layouts can be generated by changing this constraint. However, this study cannot achieve generation based on certain contour constraints. Most studies have focused on small-scale spaces such as floorplans, and there are few studies on larger-scale objects.

The results of several studies demonstrate the outstanding ability of graph neural networks (GNN) in image controllability generation. The graph structure data is similar to the functional bubble diagram in the pre-design stage of the building, where each node represents a different functional space and the connecting lines between the nodes represent the relationships between the different functional spaces. Similarly, the campus functional layout can be represented by a graph constraint structure. We hope to explore graph neural network techniques combined with campus layout generation in terms of the design logic of the campus scheme in order to achieve diverse and controllable generation results.

#### **3 Methodology**

The methods of graph constrained multiple schemes generation of campus layout is as follows:


3. Develop experimental process. The main process of experiment includes database establishment, model training, model test and result analysis.

#### **3.1 Model Architecture**

In this experiment, two deep learning models: House-GAN ++ and pix2pix, were used to generate campus functional layout and campus building layout respectively (Fig. 1).

**Fig. 1.** The idea of the experiment

The overall structure of the House-GAN ++ model is a generative adversarial network (GAN), which consists of a generator and a discriminator (Fig. 2). Generators and discriminators are constantly competing with each other to generate more realistic images. Different from pix2pix, House-GAN ++ model's generator and discriminator use a convolutional message passing neural network (Conv-MPN) [14], and input constraints are encoded into this graph structure. Conv-MPN is a variant structure in graph neural networks (GNN). It is characterized by the use of the data structure of the graph to exchange and transfer information between nodes in the graph, constantly updating the node information for the purpose of learning about the image. Since the original House-GAN ++ model did not define the boundaries of the generated objects, the generation result boundaries were uncertain. Based on the goals of this experiment, we add a boundary loss function to the original loss function so that the generated layout scheme can be constrained to a certain campus site boundary.

The pix2pix model also belongs to one of the generative adversarial networks (GAN), containing a generator and a discriminator. The generator uses a U-Net structure to encode the input image and then decode it, ultimately generating a false image that resembles the real one. The discriminator uses a PatchGAN structure, which aims to distinguish between real samples and fake samples. The two are constantly playing against each other, and if the image generated by the generator can fool the discriminator, the model can be considered to have learned the rules of the image. Using this model architecture, the machine can learn the regular logic between the input conditions and the output conditions for the purpose of this experiment.

**Fig. 2.** Model architecture

# **3.2 Training Method**

Based on the experience of the original experiment, the step-by-step generation method was determined (Fig. 3). After being familiar with the process and method of campus layout design, the campus layout generation process is disassembled. Two machine learning models are trained. The first model is to input the site contour and functional bubble connection graph and output the campus functional zoning layout; the second model is to input the campus functional zoning layout and output the campus building layout.

# **4 Database Establishment**

Database establishment includes data collection, data selecting, sample labelling, data augmentation.

#### **4.1 Data Collection**

We used manual methods to collect the campus data through a variety of means, including the portfolios of architectural design institutes, university official websites, and related books and papers.

**Fig. 3.** Step-by-step generation method

# **4.2 Data Selecting**

In order to ensure the effectiveness of the experiment, the data selected needs to follow certain rules.


Finally, 230 campus samples were selected, 30 of which were used as test samples.

# **4.3 Sample Labelling**

There are some problems in the proportion and style of the collected data, which need to be further processed before the images can be learned by the machine. Firstly, the data was scaled using the 400m track and field as a standard to ensure that the data were of the same proportion. Then we labelled the raw data through the uniform labelling rules (Fig. 4).

The process of extracting the functional bubble connection graph is shown in Fig. 5. The connection relationship referred to means that when two functional zonings are adjacent, the two corresponding functional bubbles are connected to each other. We use python to recognize the center point of each zoning, and then draws the smallest outer rectangle of each functional zoning. If the smallest outer rectangles of two partitions


**Fig. 4.** Color block labelling rules

intersect, then the center points of the two zonings are connected to each other. The graph of functional connectivity generated is the graph structure data. The nodes represent the types of functional zonings of the campus and their attributes correspond to their layout forms. The edges represent the connections between the campus functional zonings. When two nodes are connected to each other, the corresponding functional zoning of the campus is adjacent to each other.

**Fig. 5.** Extraction process of functional bubble connection graph

The data labelling points are formulated from five aspects: main entrance layout, road traffic organization, functional zoning, building layout and public space. Then we labelled the data according to the points and the actual situation of the original case. The experimental dataset was labelled to obtain 210 campus site condition data, 210 campus functional bubble connection graph data, 210 campus functional zoning layout data and 40 campus building layout data (Fig. 6).

**Fig. 6.** Partial labelling data

### **4.4 Data Augmentation**

We use data augmentation methods such as mirroring and rotation of images to increase the amount of training data. The amount of data for the training data in the first step is 1600 sets. The amount of data for the training data in the second step is 120 sets because of the deletion of the data for the incorrect building orientation.

# **5 Training and Test**

# **5.1 Training**

After conditioning the model, the dataset is input into the corresponding machine learning model for training according to the method mentioned in 3.2. This experiment combines the experience of the pre-experiment. In the first experiment, the learning rate of the model is 0.0001 and the number of iterations (n\_epochs) is 10,000. In the second step experiment, the learning rate of the model is 0.0002, the number of iterations (n\_epochs) is 800, and the input and output image channels are 3.

# **5.2 Test Results**

We tried to test the data on a variety of campus site scales. Different results were generated by changing the function bubble connection graph, some of the results are shown in the following (Fig. 7).

#### **5.3 Result Analysis**

The results show that by changing the function bubble connection graph, different layout schemes can be obtained. The model can also generate multiple layout schemes based on the same constraints.


**Fig. 7.** Partial test results of the experiments

# **5.3.1 Qualitative Evaluation**

Combining the theories and design methods related to campuses, the qualitative evaluation indicators for the experiment were set as follows: main entrance layout, campus functional zoning, road network structure, campus building layout and public space layout. We evaluate the reasonableness of the test results based on the evaluation indicators.



**Fig. 8.** Results of partial test data in the first step

3. Road network structure. The roadways between the functional zoning are continuous and complete, and most of the generated results are able to form a clear central loop. Pedestrian traffic is well connected and it is able to form a more complete pedestrian loop around the central landscape.


The grading rules are formulated from an architectural point of view and each test data is scored separately. The results were graded as A, B and C. After statistics, 83% of the results from the first step of the experiment achieved an A grade, 40% of the results from the second part of the experiment achieved an A grade, and all results from both experiments were graded B and above.

#### **5.3.2 Quantitative Evaluation**

The quantitative indicators are the diversity of the generated results, the area proportion of functional zonings, the building density and the floor area ratio.




#### **6 Discussion**

This paper explored user-constrained campus layout generation strategy from the perspective of "one-to-many generation" in combination with common design scenarios. Our experiment realized the generation of campus layouts based on user constraints, and improved the richness of the generated results. This is an in-depth study of the original experiment. Better experimental results are achieved by proposing a method for transforming graph-constrained data for campus layout images, improving the training method and modifying the loss function in the model. In exploring deep learning for design solution generation, we should give more consideration to the relationship between real design conditions and design solutions, and think in terms of the essence of design, so that the research results can solve real design problems.

In conclusion, this is a meaningful exploration to give more ideas for deep learning for layout design generation studies. The experimental results demonstrated the feasibility of improving the controllability of user input to achieve the generation of diverse results. However, the experiment is still limited by the amount of data, and there are still some data in the results that did not fully satisfy the constraints, as well as some constraints in the richness of the generated results. In the future, we will further improve this issue and hope to explore the complete campus scenario generation process. We will establish a professional technical process from site study, demand analysis, to multiple solution design, solution comparison, solution optimization and finally visual 3D solution model representation.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Lightweight and Customized Design via Conformal Parametric Lattice Driven by Stress Fields**

Fuyuan Liu, Min Chen(B) , Lizhe Wang, Zhouyi Xiang, and Songhua Huang

School of Advanced Technology, Xi'an Jiaotong-Liverpool University, Suzhou 215000, Jiangsu, China

min.chen@xjtlu.edu.cn

**Abstract.** Additive manufacturing has opened up new opportunities for materialbased design and optimization, with lattice materials being a key area of interest. Lattice materials can exhibit superb physical properties, such as high thermal conductivity and excellent energy absorption, and be designed to meet specific design objectives. However, optimizing the use of these materials requires considering geometric constraints and loading conditions. This research explores stress-driven multi-agent system (MAS) to achieve high-performance lattice infilling. The von Mises stress and principal stress are investigated as the infilling environments as they are typical failure evaluation criteria. The feasibility of these approaches is demonstrated through a case study of sport helmet design, where MAS is used to generate conformal lattice structures that meet functional and fabrication requirements. The density distribution and arrangement direction of lattice units are effectively controlled in physical fields. The results demonstrate that both von Mises stress field and principal stress field-driven methods can improve the stiffness of helmets compared to the method that only considers geometrical conformity under the same mass. The paper concludes that stress-driven lattice infilling has the potential to revolutionize material-based design and optimization in additive manufacturing.

**Keywords:** Conformal design · Lattice material-based design · Stress-field driven agent system · Design for additive manufacturing

# **1 Introduction**

Material development is crucial to optimize product performance, especially for lightweight and customization [1–3]. Additive manufacturing has opened up new opportunities for material-based design and optimization [4]. With designing the morphology of lattice units, lattice material can exhibit superb physical properties that serve different design objectives, such as negative Poisson's ratio, high thermal conductivity, good stiffness, and excellent energy absorption [5]. However, maximizing the utilization of lattice materials under geometric constraints and work conditions can be a challenge.

Loading impact and geometric restrictions are two essential factors that affect the mechanical performance of lattice structures [6]. To address loading impact, two approaches can be taken for lattice infilling: changing the relative density or population orientation. For instance, Wang et al. varied the relative density of cells to resist loading impacts [7]. Stephen et al. constructed principal stress trajectories to infill lattice material graded spatially based on lattice cell orientation [8]. When dealing with geometric restrictions, spatial tessellation is often used to discretize design domains, such as sphere packing and meshing methods [9]. Although many schemes for conformal lightweight design have been proposed, they have yet to be implemented in product design effectively. Moreover, previous studies often overlook the limitations of additive manufacturing. The mechanical performance of lattice structures is greatly influenced by the minimizing feature size, and accurate porosity control is necessary for lattice infilling.

To overcome these challenges, a multi-agent system (MAS) could be a promising solution for high-performance lattice infilling. The high adaptability of generative complexity makes it possible for MAS to generate conformal, functional lattice structures [10–12]. As an integration of numerous intelligent entities, MAS can sense different parameters and perform designated actions in physical fields [13]. For lattice infilling, the von Mises stress field and principal stress field can be regarded as perceived environments for regulating relative density and population orientation, respectively. The pore size of lattice material may be effectively controlled by regulating the movement of agents.

This paper illustrates a case study of sport helmet design in order to explore various lattice infilling methods based on stress-driven MAS. The MAS tessellates design domains into numerous cells to infill the lattice, achieving a conformal lattice structure that satisfies the functional requirements. Agents tessellate design domains under a scalar field by von Mises stress and a vector field by principal stress, respectively. The sport helmets are reformed using customized lattice material, and the feasibility of design methods is demonstrated by compressive testing to improve stiffness. The density distribution and arrangement direction of the lattice units are effectively controlled in physical fields.

# **2 Design Methodology**

This work illustrates stress-field-driven lattice infilling methods using sport helmets as examples, since porous lattice structures not only meet customized functions in practical scenarios but also guarantee breathability. This method may improve the wearing experience of users. Additionally, AM accelerates the progress of customized design and fabrication [14]. An integrated approach is proposed here to address material-driven reforming design, as shown in Fig. 1.

#### **2.1 Tailoring Free Shape Modelling and Defining the Functional Requirement**

For custom-fit sport helmets, the first requirement is to ensure a conformable interaction when users wear helmets. The second requirement is to thicken an inner shell structure

**Fig. 1.** Design framework

in a parametric manner. The method relies on 3D scanning to customize helmet lines to best fit each user's head, avoiding the limitations of standard helmets' sizes and fitting.

**Fig. 2.** Custom-fit helmet shell

A digital head model is taken as a design reference [16], as shown in Fig. 2. The morphology of the inner layer is consistent with the contour of the head model. The inner shell is scaled outward for a thickened shell. Three parameters are defined to control the size of the outer layers. The mass center of the sport helmet is regarded as the scale center, and the outer layer is scaled based on X-Y, Y-Z, X-Z plane, as shown in Fig. 3. This method enables the design of customized helmets without uncomfortable pressure points and adapts to individual differences.

The functional requirements of the sport helmet are defined according to its application scenario. Structure stiffness is regarded as the optimization target. It is assumed that the load condition of the sports helmet is based on a standard compressive test. The inner layer is numerically analyzed as it directly interacts with the head model. The analysis result is used to refine both von Mises stress data and principal stress data.

As illustrated in Fig. 4, a 1mm displacement is imposed on the inner shell at the yellow label, and blue labels are set as fixed supports. Owing to the size limitation of available 3D printers and testing devices in our lab, the helmet samples are fabricated

**Fig. 3.** Scaling shell structure

**Fig. 4.** Load and boundary condition for sport helmets

with scaled dimensions. The small displacement ensures the shell structure is in the elastic deformation. FormLab3 white resin is selected as the fabricated material. Its mechanical properties are shown in Table 1.

**Table 1.** Material properties


Static structural analysis is executed in ANSYS workbench. Von Mises stress data and principal stress data at each node are exported, as seen in Fig. 5. Both types of data are regarded as the simulated environment for the agent system in next stage.

**Fig. 5.** The results of finite element analysis **a** von Mises stress data **b** Principal stress data

#### **2.2 Field-Driven Agent System**

Multi-agent systems driven by stress field divide design domains into cells for infilling lattice, achieving a desired conformal lattice structure satisfying function requirements. In material mechanics, there are two typical stress types, von Mises stress and Principal Stress, adapting to different failure mechanisms. The former considers shear stress failure based on the Distortion Energy theory, while the latter adapts to normal stress failure according to the Maximum Normal stress theory. In this paper, the field-driven method based on both types of stresses is discussed.

#### **2.2.1 Von Mises Field Driven Conformal Design**

Von Mises stress is an equivalent stress based on shear strain energy, which can reflect stress intensity suffered by given components, as follows:

$$
\sigma\_s = \sqrt{\frac{(\sigma\_1 - \sigma\_2)^2 + (\sigma\_2 - \sigma\_3)^2 + (\sigma\_3 - \sigma\_1)^2}{2}} \tag{1}
$$

where, σ1, σ2, σ<sup>3</sup> donate the first, second and third principal stress, respectively. σ*<sup>s</sup>* means von Mises stress.

In this strategy, the notion of circle packing is introduced for conformally tessellating the shell structure. Each agent is regarded as a sphere and then disturbs the shell structure without overlapping while the size of spheres decreases as the intensity of von Mises stress increases. The sphere's center is used to construct lattice frames. The image packing algorithm developed by Daniel Pike is employed for regulating the nodes density of lattice, which is realized by Kangaroo, a plugin in Grasshopper™ in the Rhinoceros software package® (Robert McNeel & Associates, Seattle, USA). Here, it establishes a mapping relationship between brightness and von Mises stress intensity, as seen in Fig. 6.

**Fig. 6.** Mapping von Mises stress to Color

Brightness decreases as the intensity of von Mises stress increases. And bilinear interpolation is used to evaluate the *color<sup>T</sup> <sup>B</sup>* by four known color values *color*<sup>1</sup> *<sup>B</sup>*, *color*<sup>2</sup> *B*, *color*<sup>3</sup> *<sup>B</sup>*, and *color*<sup>4</sup> *B*.

$$\begin{array}{l} color\_B^T = w\_1 \times color\_B^1 + w\_2 \times color\_B^2\\ + w\_3 \times color\_B^3 + w\_4 \times color\_B^4 \end{array} \tag{2}$$

$$\mathbf{w}\_{l} = \frac{\text{Area}\_{\mathbf{w}\_{l}}}{\text{Area}\_{\text{total}}} i = 1, 2, 3, 4 \tag{3}$$

In addition, the minimum size of the sphere can be regulated for guaranteeing successful manufacturing.

The agents by von Mises stress are following the below behavior principles:


The specific design procedure can refer to [15]. Finally, all the nodes are used to generate Voronoi or Delaunay wireframes, as shown in Fig. 7.

**Fig. 7.** Sphere packing driven by von Mises stress field

#### **2.2.2 Principal Stress Driven Conformal Design**

Principal stress is a vector, which can reflect the load transfer mechanism. Brandmaier et al. introduced the effect of the principal stress direction: when the orientation of bars is along principal stress directions in the frame, each bar can suffer the minimum shear stress so as for high stiffness structure [16]. The distribution density of principal stress trajectories reflects the stress intensity and the direction of principal stress represents the effect direction of principal stress. The trajectories guide the generation of truss-based structure.

By finite analysis method, three principal vectors at each node are solved as follows:

$$\begin{bmatrix} \sigma\_{\mathbf{x}} - \lambda\_1 & \tau\_{\mathbf{y}\mathbf{x}} & \tau\_{\mathbf{C}\mathbf{y}}\\ \tau\_{\mathbf{xy}} & \sigma\_{\mathbf{y}} - \lambda\_2 & \tau\_{\mathbf{C}\mathbf{y}}\\ \tau\_{\mathbf{x}\mathbf{z}} & \tau\_{\mathbf{y}\mathbf{z}} & \sigma\_{\mathbf{z}} - \lambda\_3 \end{bmatrix} \times \begin{split} \mathbf{r}\_j^{\text{principal}} = \mathbf{0} \times \mathbf{v}\_j^{\text{principal}} \neq \mathbf{0} \quad j = 1, 2, 3 \end{split} \tag{4}$$

The σ and τ are the normal stress and shear stress at each node, the eigenvector in three directions is donated as *v principal <sup>j</sup>* .

The principal stress trajectory starts from a seed point *Pseed* on the domain, the seed point move within the stress field until the point is out of the domain. In each moving, the point will find the nearest FEA node then move along with the principal stress direction that approximates to the pervious moving direction, as below:

$$P\_{l+1} = P\_l + \Delta \cdot \nu\_j^{princlial} \tag{5}$$

*Pi* is current position of the agent, is a constant, denoted to a moving step. *v principal j* is the principal vector of the node nearest to the *Pi*.

The agent by principal stress field is following:


Here, it uses Quad Remesh, a Grasshopper component, to divide the shell of sports helmets into quadrilateral panels under the guidance of principal stress trajectories. Finally, each panel is transformed into an infilling cell by Pufferfish, an open-source plugin of Grasshopper™, as illustrated in Fig. 8.

**Fig. 8.** Principal stress driven lattice infilling

# **3 Physical Experiment and Discussion**

The abovementioned wireframes are transferred into solid model by Dendro, an opensource plugin for Grasshopper™. All optimized helmet samples are fabricated by Stereo lithography Appearance (SLA) on FormLab3 and conducted compressive tests on SanSiZongHeng universal testing machines to verify the advantages of the two methods.

**Fig. 9.** Compressive testing

The experiment aims to demonstrate whether conformal gradient lattice infilling can improve the stiffness of sports helmets, as shown in Fig. 9. The lattice sport helmet only considering geometric conformal infilling is regarded as a comparative group. A just geometric conformal helmet is generated directly by Quad Remesh component without the guidance of any curves. The displacement of 1 mm/min is imposed on all the samples and the Force-Distance curves are shown in Fig. 10.

**Fig. 10.** The force-displacement curves of all samples

Compressive tests were performed by the proposed method and the results were illustrated as shown in Table 2. The weight of all the samples was controlled at 8.00 g. A Force-Displacement curve is obtained by three points bending experiments. According to the results of experiments, the linear stiffnesses of all simples are calculated with following Eq. (6):

$$k = \frac{\Delta F}{\Delta \delta} \tag{6}$$

where, *k* is denoted by linear stiffness. *F* is a force suffered by samples, and δ is a displacement produced by the force. *F*/δ is the slope of the force-displacement curve when the linear elastic deformation happens. The displacements at 0.5–0.6 mm and their responding forces are used for calculating the linear stiffness of samples, as illustrated in Table 2. Each structure produced five samples for comparison.

By comparison, it was found that samples vM-Voronoi, vM-Delaunay, and PS-Edge increase linear stiffness by over 0.6 times over geometric conformal lattice structures. This demonstrates that gradient lattice structure benefits stiffness optimization. Theoretically, the principal stress-driven method can present higher mechanical performance than the von Mises stress-driven method. Because the von Mises-driven method prefers to solve local stress concentration and achieves a gradient distribution of lattice material, whereas principal stress trajectories take the load transfer mechanism into account and regulate both lattice density and orientation. However, the results show that the von Mises stress-driven method has the highest linear stiffness among all samples. The main reason for this is that it adds an exponential factor to reinforce the effect range of the von Mises stress field, and then make the density distribution of lattice material reasonable, as referred to (15).

Another important consideration when infilling the lattice material is its anisotropy, especially when using the principal stress-driven method. In Group 1 and Group 2, two types of lattice structures were used: PS-Edge and GC-Edge, which are edge-cubic lattices, and PS-BCC and GC-BCC, which are body center cubic lattices. The edgecubic lattice is 90° orthotropic, which allows stress transfer direction to align with the material's maximum Young's modulus, resulting in high stiffness. However, the body


**Table 2.** Samples by additive manufacturing

center cubic lattice has a 45° orthotropic, meaning that the lattice mainly suffers from bending forces, resulting in low linear stiffness.

# **4 Conclusion**

This research presents two stress-driven MAS that release the full potential of lattice material and optimize the mechanical performance of sports helmets. By dividing the helmets into cells and infilling them with lattice material in a conformal manner under von Mises stress and principal stress, the density and orientation of the lattice material can be regulated to achieve functional, lightweight, and customized designs.

Compressive testing verified that the stress-driven methods produced products with superior performance compared to designs that only considered geometric conformal design. Additionally, the methods enable products with both high performance and aesthetically pleasing natural-like structures.

The MAS driven by the von Mises stress field allows for gradient material population, while also considering the minimum size features of the lattice material for successful fabrication. The MAS driven by the principal stress field can effectively regulate the populating orientation of the lattice material for improving stiffness. These design concepts can be further integrated with a lattice library to meet diverse functional requirements and personalized user preferences.

The proposed methods have the potential for broad industrial applications, such as in Unmanned Aerial Vehicles (UAVs) and automobiles. The methods can be integrated into automatic and iterative design progress, such as genetic algorithms, to yield diverse design outputs with robust mechanical performance.

**Acknowledgement.** This work was supported by the Research Development Fund of Xi'an Jiaotong-Liverpool University [RDF-17–02-44, RDF-SP-122], Industry Research & Development Projects [RP0028, RP0029] and XJTLU AI University Research Centre.

# **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Simulation and Optimization**

# **Optimizing for Orientation in Complex Spaces**

Xuexin Duan(B) and Patrik Schumacher

Tongji University/Zaha Hadid Architects, Shanghai, China duanxuexin@gmail.com

**Abstract.** In response to the increasing demand for collaboration and knowledge exchange within Postfordist network society, both virtual and physical spaces are becoming more and more complex. Therefore the orientation within these increasingly complex and information-rich scenes becomes a problem that architectural design must address. The goal of this research is to upgrade architectural design competency in this respect by setting up a workflow for evaluating and optimizing the legibility of complex scenes. This paper introduces a novel research approach focused on the recognizability of salient interaction offerings within complex spatial settings, by using machine learning. A systematic workflow is being developed for simulations that appraise and rank design proposals with respect to the tradeoff between scene complexity and legibility. The authors explore the research through a series of simulation experiments concerned with semantic segmentation, i.e. with distinguishing and classifying relevant features in a large complex visual field. The paper first describes the method of setting up the measurement of complexity and ease of recognition, and then illustrates how a trained neural network can be used to evaluate and rank a series of design proposals (with systematically varied degree of complexity) on the basis of their recognizability. While the paper found that the hypothesis of a statistical inverse correlation or trade-off between complexity and recognizability holds, for each degree of complexity there are several design options with different degrees of recognizability. Therefore this approach allows to optimize the design of complex scenes in terms of the crucial criterion of legibility.

**Keywords:** Orientation · Legibility · Complexity · Spatial cognition · Machine learning · Design optimization

# **1 Introduction**

In response to the increasing demand for collaboration and knowledge exchange within Post Fordist network society, spaces are becoming more and more complex. Therefore the orientation within these increasingly complex and information-rich scenes becomes a problem that architectural design must address. This poses a serious challenge to human spatial perception. Research in spatial cognition, based on observation, involves a variety of behavioural measures. Beyond mere observation and performance measurement spatial cognition theory puts forward hypotheses about how people construct internal representations of space. [5, 12, 13]. The construction of internal representations becomes increasingly challenging as scene complexity increases.

As Friedman et al. [6] have argued, it is important to consider the legibility of an environment as a potentially important criterion when evaluating a post-occupancy environment. Many studies have shown that the complexity of a building plan is one of the most important factors affecting orientation in space [18, 15, 19]. However, people's spatial perceptions sometimes do not match the results from the analysis of the 2D plan. Prior research, e.g. Weisman [18] and Passini [16], had limited the study of the relationship between 'plan configuration' and navigation to corridor-type spaces. Their approach is difficult to implement in a more three-dimensional space, such as a complex atrium space. This paper extends the study of the relationship between legibility and architectural plan for atrium spaces. Our investigation of legibility is based on the first person perspective of situated human perception.

Spatial understanding can be partitioned into discrete components instead of being integrated into a single map [11, 2] One of the components is to locate and orient yourself in a space, which requires recognition of specific spatial features. In recent years, Deep Learning-based Convolutional Neural Networks have shown that trained models are able to recognize objects with human-level accuracy and speed [9, 10]. Recent research in deep-learning based semantic segmentation systems has focused on autonomous driving, industrial inspection, and medical imaging analysis [14, 7]. Architectural design has also drawn considerable attention to deep-learning-based semantic segmentation methods. For example, classifying furniture pictures based on design features [8], and comparing the visual similarity of interior design elements [4]. However, research in this area has focused on demonstrating the capability of successfully recognizing designs in the interior design field. And this study shows the potential of utilizing this technique to recognize salient interaction offerings within complex spatial settings.

The authors explore the research through a series of simulation experiments. We start with a method for measuring complexity. Then we present a method to appraise ease of orientation, and illustrate how a trained neural network can be used to do this, and thereby allowing us to evaluate and rank a series of design proposals.

# **2 Methodology**

The goal of this research is to upgrade architectural design competency by setting up a workflow for evaluating and optimizing the legibility of complex scenes. First the authors set up the formula for calculating the variety value which represents the diversity (complexity) of the destination in space. Then, using a deep-learning-based approach, the authors set up the evaluation model for ease of orientation, which appraises the level of legibility of a given space. The orientation evaluation includes three sub-measurement values: visibility, learnability, and recognizability.

#### **2.1 The Quantitative Definition of Variety**

Various research studies have demonstrated that the complexity of the floor plan influences the ease with which users can navigate within a building [15, 18, 16].This paper interprets complexity as diversity of spatial situations. In retail space, variety means many different kinds of units instead of mere repetition. Based on the K-means clustering algorithm, the measurement algorithm extracts four geometric attributes from the unit floor plan as input data: area, perimeter, shape proxy "width", shape proxy "length". Afterwards, these 4 numbers are combined to create clusters. The algorithm outputs how many types of the units can be clustered. Lastly, it will normalize the number of types as a value between 0 and 1. For instance, a floor plan with 10 units where all units are the same will result in 1 unit type, and variety value 0, i.e. this space is very simple. And when clustered into ten types, its variety value will increase to 1, which indicates a very complex space. Based on this we measure the degree of complexity of a space.

#### **2.2 The Evaluation of Ease of Orientation**

Our comparative appraisal of legibility or ease of orientation includes three aspects, namely visibility (a mere geometric precondition), learnability, and recognizability.

#### **2.2.1 Visibility**

For defining the visibility value, the authors utilize the Unity Perception package and the Unity Dataset Insights package. The workflow is as follows: First, we systematically build up an array of complex scenes. We then test/compare the scenes as follows: Step 1, place random targets. Targets represent attractions in the space, such as shops, kiosks, etc. Step 2: Place the camera viewpoints randomly within the space at eye level, so that all possible human perceptions can be sampled. Step 3: Assign a color code to all the target objects. As the simulation runs, each frame generates one view image and one labeled image. The data includes labeling information, color coding, and pixel numbers. Second, load the data and run the script. It calculates the total pixels of all the targets automatically using a Python script. So the higher the total pixel count of the targets, the more targets will appear in people's cone of vision, which means here the higher the space's visibility value. Third, normalize the data to yield a value between 0 and 1 that represents the comparative level of visibility. This is a simple, objective measure.

#### **2.2.2 Learnability**

As mentioned above, deep learning as part of machine learning algorithms has made significant progress in recent years in improving the accuracy of image semantic segmentation. In this research, the authors developed a new estimation method by constructing and training a deep learning network for semantic segmentation of architectural interior images. The authors use this first to derive a learnability value that simulates how fast a human can recognize the features of a new environment.

The measurement process includes two main parts. First, training data set generation. A collection of images and a collection of pixel-labeled images are required to train a semantic segmentation network. Instead of manually segmenting each image, we developed a custom script that uses the Unity Perception Package to automate the process. After labeling and attaching specific color coding to the main architectural elements in the 3D scene: the escalator in magenta color, the balustrade in cyan, the glazing in dark blue, etc., we can start running the simulation. In the simulation, viewpoints are generated randomly in the scene to generate different view images and semantic segmentation images associated with the view. Second, training the network. This step involves loading the view images and semantic segmentations from the previous step to the learning machine, and the goal is to find out how much time the machine needs to learn to achieve a certain accuracy in its recognition/classification of features (Figs. 1 and 2).

**Fig. 1.** View image and Semantic Segmentation image of the view.

**Fig. 2.** Semantic segmentation image output for different views

# **2.2.3 Recognizability**

Here, in contrast to comparing how fast a machine can learn and improve to discriminate the architectural elements in different designed spaces, recognizability appraises how well a computer can recognize the architectural elements after it has completed its learning cycle. As previously stated, deep convolutional neural networks are the most powerful method for training a neural network model to extract relevant parts of an image's features. However, how to combine techniques with specific domain knowledge and how to collect data has become a critical issue. To compare how well a trained machine system can recognize specified features in different designed interior spaces, the commonly used deep learning algorithm YOLO (You Only Look Once) was used because of its fast speed and accuracy [17].

In this research, we focus on the escalators as the most significant elements of interior atrium spaces. In our approach, we trained the YOLO model to detect a new custom object (the escalator) based on the trained COCO dataset (Common Objects in Context is a dataset provided by the Microsoft team that can be used for image recognition) instead of starting from scratch. Prior to deep learning training, data annotation is required. By using the unity perception package, the authors can automatically crop the region and label each image to generate a large amount of training data quickly, rather than manually labeling each image in a common way. There are 600 images prepared, 80% of the total collected data is used for training, and 20% is used for validation.

Instead of using training time data as in the learnability simulation, the detection model solves for identifying (recognizing) the escalators in the view image and in addition outputs a confidence value for each identified element as to how confident the network is in classifying the element as an escalator. The average of all the confidence values generated in a scene feeds into the overall performance measure for this scene, indicating the scene's recognisability (Fig. 3).

**Fig. 3.** Example results from detection model, in image 1 the machine detected two escalators with confidence value of 0.86 and 0.89, the accuracy value for this view image 1 is 0.875.

# **3 Prototype Development and Trade-Off Graph**

#### **3.1 Atrium Generator Prototype Setup**

The retail atrium was chosen as spatial type capable of exemplifying the problem to be investigated, namely to maintain legibility in the face of complexity. We developed a parametric atrium generator to create a set of systematically differentiated sample spaces. The atrium designs vary the void shape and the depth of the slabs up to the glazing line. The atria are radially divided into 12 segments. In the simplest case these 12 segments are equal. In the most complex case they are all unique.

The value of variety increases with the total number of different segment types. At first the atria operated across two floors above ground, later three floors. A sequence of different design iterations was generated. This sequence measured variety values ranging from 0 to 0.91. In principle, there are unlimited numbers of ways to generate designs with the same variety value. The authors selected three design iterations for each variety value. In total there are 34 design iterations with 12 different variety values (for variety = 0, there is only one design iteration as all the units are the same). For example, in Fig. 4, the variety value increases from left to right, which means the space is becoming more complex in terms of the diversity criteria.

**Fig. 4.** Plan diagram spectrum for one floor showing the how the variety value increases while the spatial scene becomes more complex.

#### **3.2 Trade-Off Graph**

The authors hypothesized that increasing the complexity will reduce the legibility of the space. Indeed, we observed this inverse relationship between complexity and legibility. We see this as a trade-off, since both complexity (delivering diversity of destinations) and legibility (affording ease of orientation) are desirable.

After generating the design iterations with a wide spectrum of variety values, we conducted our learnability and recognizability tests for each of the designs. By mapping all the values for variety first against the obtained learnability and recognizability scores, we get the maps in Fig. 6. Based on statistical theory, when the R2 value is bigger than 0.6, this means the two parameters have a strong correlation. We also can see from the mappings that for each of the variety values, despite the overall correlation holding, there are several designs with different legibility (learnability/recognizability) scores, i.e. for each visibility value there are relatively better performing designs we might select, i.e. a design with both high complexity and relatively high legibility.

#### **3.2.1 Learnability Trade-off Graph**

With the same 34 design iterations spread over 12 design variation values, we executed the learnability experiment to measure how long it takes for the machine to achieve 95% accuracy. For instance, when the variety value = 0. It only took 45s for the machine to learn to recognize the architectural elements and reach an accuracy of 0.95. In comparison, a design with high variety (variety = 0.91), requires 134s of learning to reach the accuracy of 0.95. After training for all the design iterations, we obtained the graph in Fig. 5a which also shows that there is a strong correlation between variety(X axis) and learnability (Y axix). The R2 value is 0.7461, i.e. significantly above 0.6.

#### **3.2.2 Recognizability Trade-off Graph**

Instead of comparing the time taken to reach a certain accuracy between different design iterations, the recognition experiment uses the same trained neural network to test the recognition accuracy with different design iterations. For the same series of designs, 50 images were chosen from each of the 12 design variety brackets. In total, there were 600 images constituting the training dataset. After training, the neural network can achieve an accuracy above 0.8 for the validation set.

After building the neural network, the next step was to test the recognition performance of the neural network for the different design options with different degrees of variety. To test the different designs, 10 new perspectival view images for each design have been fed to the neural network. We then obtained the average accuracy figure for each design, which for us represents the recognizability value of this design. For instance, when the design variety equals 0.00, the average accuracy is 0.84 for the 10 images, and when the design variety equals 0.91, the average accuracy dropped to 0.66. After we run the recognition test for all the options, we get the graph shown in Fig. 5b. We can see from the output figures that the distribution of accuracy correlates with the variety value. This indicates that the recognizability value has a strong correlation with design complexity.

**Fig. 5.** Variety-Learnability (**a**), Variety-Recognizability (**b**) trade-off graphs

Both learnability and recognizability were explored via neural networks trained to semantically segment images of a complex scene and classify features like escalators. According to these experiments, there is a strong inverse correlation between variety and legibility, appraised in its aspects of learnability and recognizability. Since for each variety measure we can generate and test multiple designs, we can use this process to optimize the trade-off between variety and legibility.

#### **4 Further Design Iterations**

#### **4.1 Learnability Experiment**

It has already been mentioned that there are an infinite number of ways to generate a design for a particular variety value. Our goal is to evaluate and optimize in order to find the most legible solution. The authors compared three design options with similar plan configurations and the same variety value of 0.62, but with changing the slab edge curvature. Option A has a large curvature on the slab edge, Option B has a faceted slab edge, and Option C has a gentle curve on the slab edge. According to the learnability tests, option C exhibits a high learnability value. This is in line with our intuitive expectation that the smooth version will perform better than the faceted version (because its generates less visual noise).

**Fig. 6.** Three design iterations' legibility comparison. In option C, the visibility value is 0.745, while the other two are 0.713 and 0.736, and in option C, the training time is 3:45 while Option B's training time is much longer as it is 5:23, and Option A is fairly close as it is 3:54.

#### **4.2 Recognizability Experiments**

In addition to the prototype experiments, the authors also examined the methods on a real design project. It is a 5-floor atrium space in a mixed-use (retail and cultural) complex. Using two design iterations as examples, we applied our recognisability appraisal method. When all the necessary functional program plan layouts have been met, it is typically depending on the architect's intuitive qualitative appraisal to choose a number of design option that meet the programme criteria. In contrast to this subjective choice, our method allows for reproducible selection based on a quantitative process that can be critiqued and improved upon.

The evaluation and optimization process was applied as follows: first, we randomly selected 10 viewpoints to get 10 view images from each of the designs. Then we input these images to the neural network (trained on the proto-types) to get the escalator recognition performance with accuracy values for each image. Then we averaged all the accuracy values to get an overall recognizability value for the design option in question. Any mistaken detection is taken into account with its confidence value as negative value. Due to mistaken identifications both options' recognizability values were rather low, but this measured difference is significant. For option 1 it was 0.277 and for option 2 it was 0.023 (see Fig. 7). It was interesting that the network was performing at all in a space that was quite different from the training set. In any event, absolute values matter little. The purpose of our method is to get a ranking that guides selection. The result suggests option 1 is better in terms of recognizability. From our intuitive point of view, option 1 also looks more legible. This illustrates how a neural network can be used to evaluate and rank a series of design proposals based on their recognizability.

The evaluation process is quite efficient and easy to operate. You can upload either a screenshot or a rendering of the design space. As a result, a value of recognisability will be output for this space. Many questions and problems remain to be addressed. For instance, when feeding new images that substantially deviate from the original training

**Fig. 7.** in image 03 of option 1, the confidence value outputs are 0.76, 0.77, 0.91, but it missed some escalators, so the accuracy value is 0.407 (= (0.76 + 0.77 + 0.91 + 0 + 0 + 0)/6). And for option 2, it missed out on all the escalators, and also incorrectly "recognized" some balconies as escalators, so the accuracy value for this view image of option2 is –0.246 (= (–0.61–0.62 + 0 + 0 + 0)/5).

set, overall accuracy is not high. Hence, it will be necessary to expand the database with more interior images of atriums in the future in order to enhance the accuracy of the data. Second, the current neural network is only trained to recognize the escalator as a key element of the atrium. However, it is possible to extend it to incorporate other key architectural elements as well (Fig. 8).

**Fig. 8.** Example outputs from detection model tested on the photorealistic renderings.

# **5 Conclusion and Future Work**

The objective of this research is to provide a methodical way of posing the issue of legibility. Our work so far only maps out an initial sketch schematic for the operationalisation of the legibility criterion in its relation to complex scenes. The atrium typology is just one scenario among many. This method setup can be abstracted and generalized, and then tailored to support all urban, architectural and interior design processes. Both complexity (spatial diversity) and legibility (ease of orientation) are considered positive characteristics. While the hypothesis of an inverse correlation or trade-off between complexity and legibility holds, there are several design options with different degrees of legibility for each level of complexity. The methodology can thus help find the sweet spot in the trade-off between these two requirements, or else we can take a certain level of complexity as given, and then identify design options that maximise legibility without compromising complexity. In this way, the design of complex scenes can be optimized according to the crucial criterion of legibility.

# **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **A Virtual Reality Window View Evaluation Tool for Shading Devices and Exterior Landscape Design**

Wentao Zeng(B) and Hanyi Zhang

College of Environmental Design, University of California, Berkeley, CA, USA wentao\_zeng@berkeley.edu

**Abstract.** The window view is an important part of the daylighting design. The current window view analysis based on daylighting metrics does not respond well to user preferences. This study uses an office with a courtyard in Berkeley, CA, USA, as a case study to create a virtual reality-based window view evaluation tool and workflow to analyze the impact of different types of shading devices and different levels of exterior landscapes on user perception. This tool combines quantitative data based on daylighting metrics and users' subjective and physical responses with qualitative analysis based on user feedback and preferences. A twoway ANOVA was conducted in the study to demonstrate that the independent and interactive impacts of shading devices and exterior landscapes on user perception and satisfaction. The results show that users prefer shading types that block less of window views even though they may cause a higher probability of glare. Besides, advanced landscapes tend to enhance user satisfaction with shading devices. This new window evaluation method will help architects make more comprehensive decisions in shading device type selection and exterior landscape design.

**Keywords:** Window view · Shading device · Exterior landscape · Virtual reality · User perception and preference

# **1 Introduction**

The study of daylighting is an important part of architectural design. Architects and designers often overlook the quality of the occupant's view, an essential analysis factor of daylighting design. The research indicated that reasonable window views can reduce human physical and mental discomfort [1]. Moreover, creating a connection between landscapes and humans by enhancing window views has been proven to positively impact the health of the occupants [2]. Thus, daylighting and window views are increasingly being integrated to consider the impact on occupant health and well-being. As a medium between the interior daylight environment and the exterior landscape, shading devices must balance daylight performance and window views. Appropriate shading devices have been shown to optimize multiple aspects of daylighting, such as daylight distribution, glare, and views [3]. The selections of shading devices and landscaping in the previous studies rarely considered end-user preferences and were mostly based on daylight simulation results and subjective design by architects. Thus, to design window views conducive to human health and sustainability, the selection of shading devices and the design of the landscape should accommodate the visual comfort and preferences of users while satisfying daylight standards.

Virtual reality (VR) as an excellent medium can combine simulation-based daylighting analysis with user-based immersive analytics to integrate daylighting metrics and visual preferences of users in the early stages of the shading device and landscape design. VR is becoming an effective alternative for the evaluation of interior visual environments because it has been proven to be superior to video and pictures for subjective perception, and it allows for controlling selected variables, analyzing causal relationships, and saving time and costs spent on real building measurements [4, 5]. Abd-Alhamid et al. [6] confirmed the importance of the information content seen in the window views by analyzing the observation data and feedback from users at different locations in the VR scene and showed that the design of the window view has significant implications for the health and well-being of building occupants. Chamilothori et al. [7] combined VR and wearable biometric devices to study the effect of different shading façades and scenarios for user vision. The results showed that the different patterns and geometries of the shading façade influenced the users' subjective visual evaluation and physiological responses. Lee et al. [8] introduced a method for evaluating view clarity through VR. The study revealed that the geometry and material of the shading system can affect the clarity of the exterior landscape to the degradation of the quality of the view. To sum up, VR has a strong potential to be used as a tool to connect user perception and architectural research. Therefore, more and more architects and designers have the opportunity to gather information about end-user requirements and preferences for window views through VR to make more informed decisions during the preliminary design stage.

The research focused on shading devices and exterior landscapes as the two main points of attention in the window views study. Previous analysis of window views is typically based on a relatively elaborated assessment of the LEED v4 Quality Views (QV) credit [9]. However, this evaluation metric does not require all view criteria to be met the credit, and therefore results in window views that are often low quality and do not respond well to real user feedback [10]. Moreover, other visual perception effects such as glare and thermal discomfort are not taken into account when studying the window views. At this time, there are no established methods to guide designers and researchers in investigating users' perceptions of window views. Based on this, this study proposes the following questions in different stages of workflow:


This study aims to develop a window view evaluation tool and workflow that combines subjective and objective analysis methods. Firstly, the method from daylight models to immersive virtual environments was studied, and the method of constructing a more comprehensive VR window view analysis tool was summarized. This new evaluation tool was then used in the design of shading devices and exterior landscapes for an office in Berkeley, California, and its application to window views perception and feedback for users were explored. This VR window view evaluation tool allows architects to combine daylighting metrics and user preferences at an early stage of design to comprehensively compare and select options for shading devices and landscapes. In addition, the VR window view evaluation tool proposed in this study has been assessed by usability and universal applicability and can be applied to other types of spatial analysis.

### **2 Method**

The primary goal of this study is to propose an innovative VR window view evaluation tool and workflow. Thus, the impact of shading devices and exterior landscapes in window views was studied through immersive virtual environments to help architects find a balance between daylight performance and window views. The study uses an office with a courtyard in Berkeley, CA, USA, as an example. Rhinoceros 3D and Grasshopper were used to create a model an office model with shading devices and exterior landscapes, which allowed defining types of shading devices and different levels of exterior landscapes as parametric variables. Daylight simulation and analysis are completed by embedding the required weather files and material data through ClimateStudio to select shading devices that meet daylight requirements. Moreover, the Radiance Render function of ClimateStudio creates scenes of 360 high dynamic range renderings (HDRR) with different shading devices and landscapes. The VR scenes were created by importing high dynamic range renderings into Unity and adding an interactive interface to complete the user evaluation system. Finally, daylighting metrics, user feedback, and physical responses to different VR scenes were compared and analyzed through case studies to help the architects select the appropriate shading devices and exterior landscapes for the office space. Figure 1 shows the research process and methods of this study.

#### **2.1 Experimental Model**

The research uses a south-facing office with a courtyard on the first floor of an office building in Berkeley, CA, USA as the case study. The office's length, width, and height are 5 m, 3 m, and 4 m respectively. Moreover, the office has an exterior courtyard without landscaping with a length and width of 15 m and 10 m. In addition, the office has only one window on the south wall, which is 2.9 m in length, 1.6 m in height, 1m in distance from the floor, and with a wall-to-window ratio of approximately 40%. The weather type in Berkeley is a Warm-summer Mediterranean climate (Csb in the Köppen climate classification) with long, mostly sunny summers. Thus, the architect needed to choose appropriate shading devices and exterior landscapes by analyzing window views to create a comfortable daylighting environment. Based on information from the site,

**Fig. 1.** The workflow specified for the research.

the office was 3D modeled by Rhinoceros 3D and Grasshopper for daylight simulation, system development, and window view analysis (Fig. 2).

**Fig. 2.** 3D modeling based on real environment: real office space (left), 3D model for case study (right).

Different types of shading devices and external landscapes were set as independent variables in this study to investigate the comprehensive effect of user responses in window views. The space size, wall-to-window ratio, and materials of the case model are fixed in the analysis. Figure 3 shows the three levels of exterior landscapes created in the study: a no landscape with brick-paved ground, a normal landscape with partial grass and shrubs, and an advanced landscape with more trees and artwork. Moreover, the study selected five types of shading devices in pre-experiments using Grasshopper and ClimateStudio to meet the requirements for spatial Daylight Autonomy (sDA), i.e., the interior space receives at least 30fc of daylight for at least 50% of the workday [11]. Six types of shading scenes including no shading, low-density horizontal louvers (LH Louvers), high-density horizontal louvers (HH Louvers), low-density vertical louvers (LV Louvers), high-density vertical louvers (HV Louvers), and special egg crate louvers (EC Louvers) were set up in the study to investigate the effect of shading type and density on the visual comfort of the users (Fig. 3).

**Fig. 3.** Three different landscape levels and six different shading devices for window view analysis.

#### **2.2 Daylight Model**

The daylight model integrates 3D models, weather data, and materials for window view simulation and rendering of office scenes with different shading devices and exterior landscapes. The EPW weather data files for Berkeley were imported into the daylight model through ClimateStudio using occupancy times from 8 a.m. to 6 p.m. The materials used for simulations and renderings of the office, shading devices, and landscapes are all from the ClimateStudio materials library (Table 1). Daylight metrics were used as supplementary information for window analysis to help architects and users understand the impact of different shading devices and exterior landscapes on interior daylight performance and glare protection. In the study, sDA was used to analyze the annual daylight performance of the space to evaluate different types of shading devices. The analysis grid used to calculate the sDA was located at a table height of 0.73 m, with the sensor points spacing was 0.6 m and their distance from the walls of 0.5 m (Fig. 4). Moreover, the study used Spatial Disturbing Glare (sDG) based on the Daylight Glare Probability (DGP) metric to analyze the annual average glare of the space, i.e., the percentage of space that experiences Disturbing or Intolerable Glare (DGP *>* 38%) for at least 5% of occupied hours. The analysis grid for the sDG metric used the same sensor spacing as the daylight analysis grids. The default view was located at 1.2 m off the floor (eye height for a seated observer) and the calculation was based on hourly DGP values for eight different view directions at each position in the space (Fig. 4). As a common assessment method of window views, QV credits in this study were always at 100 due to the area of the office (length less than 7.5 m). Thus, the window views analysis method using QV could not be used in this project to investigate the impact of different shading devices and exterior landscapes in more detail. Radiance Render of ClimateStudio was used in this study to simulate the 360 HDRR by daylight model used to create the virtual scene for the window views analysis. The time of the simulation for the scenes was chosen at the highest sDG value of the year (12.30 pm on December 21), and the rendering position P was 2 m from the window at a height of 1.2 m off the floor (Fig. 4).


**Table 1.** Optical properties for objects in the simulation.

**Fig. 4.** Example of sensor points with views used for sDA simulations (left), sDG simulations (center), and window view rendering (right).

#### **2.3 Generation of VR Window View Evaluation Tool**

The VR window view evaluation tool created immersive virtual scenes for window view assessment by embedding 18 different 360 HDRRs in Unity3D (including a combination of 3 levels of exterior landscapes and 6 types of shading devices in the experimental model). The tool had three main functions to assist users in better completing the window view evaluation. Firstly, a scene-switching function was created to help the user randomly change to the next scene after completing the evaluation of one scene (Fig. 5). Secondly, as glare is an important factor in window view evaluation, the system added a glare observation function (Fig. 5). The system showed the area where the glare existed in the VR scenes by the partial false color (pink area *>* 2000cd/m2). Compared to the full false color indication of glare, the partial false color only indicates the glare area to allow the user to intuitively understand the luminance situation in the scene and help the user to complete a more comprehensive window assessment. In addition, users can record their emotions and satisfaction with the different shading devices and exterior landscapes through the evaluation function after completing the observation of each scene (Fig. 5). As listed in Table 2, the evaluation function consisted of 6 questions, all of which are measured on a seven-point Likert scale (mostly 1 = fully disagree and 6 = fully agree). The data collected by the system helped the architects better understand end-user preferences and feedback.

**Fig. 5.** The three main functions of the VR window view evaluation tool: scene-switching function (left), glare observation function (center), and evaluation function (right).


**Table 2.** Evaluation questionnaire for window view study.

#### **2.4 Experimental Design**

Instead of traditional physical window view analysis, the study was observed and evaluated through the immersive virtual environment of the VR window view evaluation tool to analyze users' perceptions and preferences for different shading devices and exterior landscapes. The headset used for VR window view observation in the study was an Oculus Quest 2 with a field of view of 100°, a resolution of 1832 × 1920 pixels per eye, and a refresh rate of 90 Hz. The study created a VR environment through Unity3D that can be used for observation of Oculus devices and interaction with the system through controllers. Moreover, a total of 30 participants (15 males and 15 females) participated in this study. Participants were limited to a range of 20–32 years (mean age: 23.8, SD = 3 years). The study used a within-subject experiment design i.e. each participant was tested on the same 18 scenes to eliminate individual differences between participants. In addition, participants were asked to test individually in a real office space. After confirming familiarity with the equipment and experimental procedures, participants were exposed to neutral scenes and tested for basal heart rate in VR, and then a series of scenes were observed in random sequences. While observing the scene, participants were required to use the glare observation function to identify areas of glare in the window and to receive 30 s of heart rate monitoring (Fig. 6). At the end of each scene observation participants were asked to complete the evaluation questionnaire to collect participants' emotions and satisfaction with the shading devices and exterior landscapes in the window views. The average observation and assessment time for each scene is about 2 min. After completing a window view evaluation of a scene participants could switch to the next scene until they completed 18 scenes. Upon completion of the experiment, participants were requested to complete the feedback on window views and system usability. Furthermore, all participants provided written informed consent before the study and were compensated for their participation.

**Fig. 6.** Participants performed window view evaluation and monitored heart rate in VR scenes.

# **3 Results Analysis**

#### **3.1 Daylight Analysis**

Although the main focus of this study is on window view assessment, daylight analysis as a basis for window view analysis helps architects and users understand the daylighting performance and glare probability of interiors with different shading devices and exterior landscape conditions. As shown in the daylight simulation results in Fig. 7, the five different shading devices in the study kept the annual sDA at 55.0–72.5%. Among them, LH Louvers and LV Louvers had the best daylight performance, with an average annual sDA of 71.3%. Moreover, the five different shading devices reduced annual sDG by 19.4–38.8% compared to the No Shading scene (Fig. 7). HH Louvers demonstrated the best glare protection with an annual sDG of 0.6%. Furthermore, compared to No Landscape and Normal Landscape, Advanced Landscape decreased the DGP of different shading devices by an average of 11.4% in the scene on December 21 at 12.30 (Fig. 7). Daylight analysis proved that Advanced Landscape, which has tall trees and artwork, has an impact on glare protection for users. In addition, the daylighting metrics provided basic information for participants in the window view evaluation and additional support for the architects in the window view analysis.

**Fig. 7.** Daylighting metrics in different window view scenes.

#### **3.2 Window View Analysis**

The study analyzed 30 sets of scoring data and physical response data obtained from the VR window view evaluation tool by experiments. The Kolmogorov-Smirnov test showed that the data for all dependent variables were normally distributed, therefore a two-way ANOVA was used to determine the independent and interactive impacts of two factors, the level of exterior landscapes and shading types [12]. Statistical analyses of data were performed in Python, using the toolbox from Pandas, Matlibplot, and Statsmodels. For the multiple independent hypotheses, a Bonferroni-corrected significance level α of 0.0014 is used for the within-subject factor analyses. Thus, the level of the strong effect of significance is 0.001.

#### **3.2.1 Subjective Responses**

In this section, user perceptions and feedback regarding shading types and exterior landscape levels in window views were investigated. A two-way ANOVA was performed for each dependent variable to detect the effects and interactions of shading types and exterior landscape levels on users' emotions. The statistical analysis showed a significant effect of shading devices and exterior landscapes on perceived pleasure, interest, and excitement (SSlandscape\_pleasant = 611.411, Plandscape\_pleasant *<* 0.001, SSshading\_pleasant = 149.817, Pshading\_pleasant *<* 0.001; SSlandscape\_interesting = 549.081, Plandscape\_interesting *<* 0.001, SSshading\_interesting = 182.326, Pshading\_interesting *<* 0.001; SSlandscape\_exciting = 601.893, Plandscape\_exciting *<* 0.001, SSshading\_exciting = 180.542, Pshading\_exciting *<* 0.001). The results of the analysis supported the research variables that different shade types and levels of landscape affect participants' responses to the degree of pleasure, interest, and excitement of spatial perception (Table 3). The study demonstrated that participants probably were more pleasant, interested, and excited in advanced landscapes or types of shading devices that blocked less of window views (Fig. 8).

A two-way ANOVA was conducted on participant satisfaction data to investigate the independent and interactive impacts of independent variables on shading device satisfaction (SDS), glare satisfaction (GS), and exterior landscape satisfaction (ELS), (SSlandscape\_SDS = 140.011, Plandscape\_SDS *<* 0.001, SSshading\_SDS = 921.422, Pshading\_SDS *<* 0.001; SSlandscape\_GS = 47.004, Plandscape\_GS *<* 0.001, SSshading\_GS = 799.637, Pshading\_GS = 0.008 *>* 0.001; SSlandscape\_ELS = 629.559, Plandscape\_ELS *<* 0.001; SSshading\_ELS = 173.837, Pshading\_ELS *<* 0.001). The results of the study showed that the interaction of different levels of landscape and different types of shading devices was significant in terms of shading device satisfaction, but not in terms of glare impact and exterior landscape satisfaction (Table 4). Participants tended to score higher on shading device satisfaction and diminished scores on glare discomfort in more advanced landscapes (Fig. 9). Moreover, when the landscape was more obscured by some type of shading device, participants tended to give lower scores on this type of shading device (Fig. 9).

#### **3.2.2 Physiological Responses**

The study used a two-way ANOVA to analyze the independent and interactive effects of shading devices and external landscapes in the window views on physiological responses



**Fig. 8.** Interaction plot for average rating of pleasant level (left), Interesting level (center), and exciting level (right) was perceived in different scenes.

(Table 5). The results showed no significant interaction between shading types and landscape levels (Pshading\_landscape = 0.008 *>* 0.001). However, the effects of both shading devices and exterior landscapes on heart rate were significant. Figure 10 shows that the advanced landscape scenes had the lowest heart rate differences, meaning that participants were probably in the calmest emotions at that time. Moreover, for scenes with HV Louvers, excessive view blockage and glare may have contributed to the high heart rate differences of participants, meaning that participants may have had uncomfortable emotions at that time (Fig. 10).

#### **3.2.3 User Feedback**

Besides the scoring data of window views, the experiment also requested feedback from the participants through questionnaires. Questionnaires are an effective way to collect information on user preferences and behaviors. The study obtained qualitative data on user preferences and usability assessment for the VR windowing evaluation tool through user feedback (Fig. 11). Regarding the window views provided in the study, most users stated that "shading devices that block less of view give me a better visual experience", "I definitely prefer advanced landscapes, it makes me physically and mentally happy", and "I think the combination of low-density louvers and advanced landscapes is my preferred window views". However, a small number of participants indicated "I personally dislike excessive direct daylight, so I prefer shading devices with good shading, even if most of the view is blocked", and "I prefer normal landscapes with only grass and shrubs, such exterior landscapes make me peaceful". The analysis of user feedback showed that users are more concerned about getting relatively unobstructed window views than about glare. Therefore, LH Louvers and LV Louvers with more open window views are the preferred types of shading devices for users. Furthermore, the advanced landscape became the preferred level for most participants. User feedback helped architects to understand user preferences more intuitively and select appropriate shading devices and exterior landscapes.

User feedback also provided a system usability assessment for the study (Fig. 11). On the positive side, users commented that "it provides me with a good visual immersive experience", "the scenes are rendered realistically" and "the system is impressive, comfortable to observe and the interaction menus are responsive". However, users also reported that "I would like to have more interaction with things in the environment", "I wish I could move around in the scenes", and "some parts of the scenes are not clear due to overexposure". Participants' feedback on the system mostly focused on improving



**Fig. 9.** Interaction plot for average rating of satisfaction with shading devices (left), glare (center), and exterior landscapes (right) in different scenes.

**Table 5.** Results of two-way ANOVA testing the effects and interaction of landscape levels and shading types on heart rate.


**Fig. 10.** Interaction plot for average heart rate differences in different landscape levels and shading types.

hardware technology and enhancing interactive functions. The user feedback will inform and assist in the future development of the VR window view evaluation tool.

**Fig. 11.** Excerpts from questionnaires of user feedback.

# **4 Conclusion**

The evaluation of window views requires consideration of several aspects, including shading devices, exterior landscapes, wall-to-window ratios, window materials, observation locations, etc. This research focuses on shading devices and exterior landscapes in the window view evaluation. The VR window view evaluation tool and workflow developed in this study can help architects to comprehensively assess window views and apply them to the design of shading devices and exterior landscapes. The originality and value of this window view research are as follows:


The results of the window view analysis showed the effects of different shading devices and landscape levels on user emotion, satisfaction, and physical responses. Most users preferred shading devices that block less of the view, such as LH Louvers and LV Louvers, even if they have a higher DGP. Moreover, most users preferred the advanced landscape, which tends to increase user satisfaction with shading devices. Furthermore, the user feedback presented in the study can help users to select shading devices and landscape levels that have specific preferences. These analyses of user preferences helped architects to make more comprehensive window view evaluations and design decisions.

The scenes used in the VR window view evaluation tool for this study are simulated based on the case model, which can be replaced with different window view scenes through the design workflow proposed in the paper. Moreover, the system has only one fixed observation position that does not allow for a multiple-perspective window view observation. Therefore, the collection of user perceptions and feedback may be defective. A movable immersive observation environment and more comprehensive physical detection equipment may be added in further studies to complete a more comprehensive window view evaluation tool.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Virtual Reality Architecture Teaching Application Based on Unity Platform—Taking a Small Architect's Metaverse Application as an Example**

Yihan Lu, Zhouyu Zhang, Yaoping Zhang, and Li Li(B)

Southeast University, Xuanwu, Nanjing 210096, China 101012053@seu.edu.cn

**Abstract.** Children's architectural education is mainly built by assembling existing prefabricated blocks. However, most of the existing architectural education projects focus on PC applications, which are important for transforming architecture into practical results. Taking the achievements of the VR architectural knowledge popularization education project as an example, this paper demonstrates the application of the combination of virtual reality technology and architectural education in the future architectural design education for children. The app adds a follow-up connection to reality that is missing from traditional VR experiences. First of all, this application built a virtual reality building on the unity3d platform, controlled vr equipment through the OpenXR library, designed a program to build a house with prefabricated modules in the application, and finally exported the virtual results as DXF files to the laser cutting equipment, and assembled them in reality. It retains the interest and diversity of the virtual meta-universe and can increase the authenticity of the combination of virtual and real life in the education links to improve hands-on ability. This application through the students in a three-dimensional virtual environment to experience the process of building, but it also can use laser printing technology to transform students' learning results into real results in order to realize the application of virtual reality technology in architectural education.

**Keywords:** Virtual reality · Architectural education · Block cutting and assembling · Construction logic

# **1 Introduce**

Building education for children is a common type of education. Most of the educational applications of VR are directly applied to architecture exhibitions or purely virtual world construction, lacking the process of hands-on practice, and the results have not been turned into reality. Based on relevant literature research results and user research, this paper introduces the logical framework of a VR building construction system and, on this basis, develops a set of VR building teaching application design frameworks and design strategies, which explores a path for the combination of virtual reality and children's architectural education in the future. Teaching for children aims to enhance the sense of experience, exercise practical ability, and enhance the understanding and interest in architectural knowledge. Explore a VR teaching system platform based on virtual reality technology to improve the diversity and sense of the experience of architectural education (Fig. 1).

**Fig. 1.** Application scenario screenshots

# **2 Background**

#### **2.1 Current Status of VR Education Development**

At present, elementary school architecture popular science education mainly stays at the display level of traditional multimedia. Two-dimensional image and audio-video display cannot effectively enhance the interaction between teachers and students, students' sense of participation is not high, and the degree of knowledge absorption and other aspects of learning performance is not good. The pattern between teachers and students is that one teacher corresponds to many students, and problems can not be solved in time, as well as how to save the architectural design, construction, and assembly games. The application of virtual reality technology in a wide range of fields in China, but the penetration rate of application in education is still very low. First, because of the high cost of virtual reality technology equipment and manpower; Second, public awareness is not enough. [1] For teenagers, the lure of new technology is very strong. Touchstone Research, an online marketing research company, surveyed 500 teenagers between the ages of 10 and 17 and released statistics on the awareness of virtual reality technology: 79% of people have heard of virtual reality technology, 68% of people have gradually learned about it, and 47% of people have some or a deep understanding of virtual reality technology. Therefore, the prospect of virtual reality technology among teenagers is still extremely optimistic. Many teenagers aged 10–17 are very interested in virtual reality. "VR virtual reality devices have a far greater impact on minors than TV, computer and other products." The way people perceive the world is being gradually changed by virtual reality technology, especially for students between the ages of 10 and 17; This technology has a disruptive impact [2].

# **2.2 Introduction of VR Teaching Mode**

Virtual reality teaching is considered an emerging teaching method in the education curriculum, and multiple studies have shown that virtual reality technology has a positive impact on student's academic performance and motivation. Most of the applications of virtual reality technology (VR technology) in primary education have focused on describing and evaluating findings in terms of effectiveness, user experience, usability issues, and student learning performance in the process of primary instructional design [3].

Architecture is a subject that requires people to feel scale. In the process of learning, it is mainly necessary to observe and understand scale, colour, material, and proportion through the eyes. The world of virtual reality can also provide a real experience of scale, get rid of the limitations of the physical world, and present a rich world through a machine.

### **2.3 Feasibility Analysis of VR Education**

1. Upgrade of virtual reality technology

Now virtual reality devices can operate independently, free from the constraints of cables, which also liberates the constraints of environmental sites and makes it possible to teach small classes.

2. Hardware price reduction and popularization

After the acquisition of Oculus, Meta launched the Oculus Qust2 device, which is priced at \$299, enabling ordinary consumers to purchase experience. There is also Pico brand benchmarking in China, and VR devices are becoming increasingly lightweight, enabling more and more ordinary people to have access to this technology.

3. Application of multimedia technology in primary and secondary education

At present, many primary and secondary schools are promoting comprehensive quality education, so with the help of virtual reality technology, students can experience the process of various professions, lay a good foundation for relevant majors, and cultivate interest. And some science experiments with safety risks can bring immersive real experiences through virtual technology.

# **3 Methodology and the Main Procedure**

Our approach focuses on the underlying building of applications and effective integration with VR devices. Consideration is convenient for people to learn and understand quickly.

Build the building application platform and realize the building function. This study combines the intersection of computer science, architecture, and education.

On the application development side, unity3d was selected as the main platform of the whole process operation, and C# language was selected as the programming development language. First of all, I built the module unit in rhino modelling software, exported the module into the unity platform in obj format, and then integrated the model building system module management, DXF file export and other functions on the unity platform.

After the model is built, obtain the generated cutting file and import it to the cutting device. Assemble the cut plates (Fig. 2).

**Fig. 2.** The main steps

#### **3.1 Reality to the Virtual —VR Application Platform Design**

#### **3.1.1 Build the Bottom of the Building System**

Firstly, a 2 m\*2 m cell grid system is established. Grid cells are used to control where blocks are generated. In unity's GridManager, you can enter the desired build width and height. The BlockManager script can adjust the size of the module according to different input parameters, such as a Block 3 m high \*2 m wide \*0.2 m thick.

On the data structure of the GridSystem (Fig. 3), each grid is named GridX and GridZ, respectively, according to the x-axis and z-axis of unity's coordinate system. The grid data structure is divided into three layers, and the GridX of the same row is added to each row according to the row. As shown in the figure below, each GridX record has the column information and the corresponding position of the name, and then multiple rows are added to each layer. The GridX of each layer is then added to a collection called the three-tier list hierarchy. This allows the aspect to read the corresponding position in the program through the sequence.

In the step of placing the block model, the raycast (Fig. 5) method provided by unity3d was used to obtain the grid target hit by the laser ray issued by the VR controller, and the block was created under the object.

Next, the grid is generated based on blocks that can be built in the next layer. After a layer is built, a new layer of GridX and GridZ is generated based on the GridX and GridZ rows and columns of the first layer of blocks. This allows the building to gradually decline from the lower level, which is also in line with the logic of building in a gravity environment. At the same time, the GridCell class is created to record every row and column of GridX and GridZ. This is easy to find the corresponding position in the

**Fig. 3.** Data structure of the grid and the corresponding hierarchy diagram representation

back. During construction, it will detect which grid raycast hits, and after confirming construction, the block will be generated under the grid object. Therefore, it only needs to find the grid with child objects to get the corresponding grid.

Then, rotation and determination of positive and negative rotation (Fig. 2), when each block is placed, there will be a difference between positive and negative, as well as a difference between length and width. Therefore, a record will be made for the direction of the grid and block when it is generated. When a block is generated in the grid, it will judge whether the direction is aligned with the long side so as to ensure that the long side is aligned with the long side. As shown in the figure below, the grid itself will have four directions when it is generated, and the block will also have directions when it is placed. (Fig. 4) matches it in the same direction.

**Fig. 4.** Schematic diagram of the orientation adjustment algorithm of the block enclosure

Finally, in automatic cover determination, after building a floor, the last module returns to the starting position to form a closed ring to generate the floor cover of this floor, based on the range of blocks of this floor, through the GenerateNewFloor method in the GridSystem script. The principle is to detect the maximum and minimum values of blocks in the same column or row to generate the floor within this range. Also, In GridSystem 'CreateStandFloor' method also generates floor non-builds that can stand (Fig. 6).

**Fig. 5.** Camera raycast gets the target grid and generate the block under the corresponding grid

**Fig. 6.** The grid diagram in the multi-layer process is built

#### **3.1.2 Modular Unit Design**

How to reduce cognitive difficulty, improve the richness and increase the fun of architecture from the perspective of children?

In the present architectural design, the modular building has developed to a certain mature stage. In children's toys, building blocks can also be considered modules, so in the early design, there was a choice between voxel unit blocks like Minecraft and modular boards like the SIMS [11].

In the process of module design, the module is also considered.3 m is the common base height. In terms of width, the final consideration is given to both efficiency and free openness. 2 m is selected so that there can be more designs for door and window modules, and rich buildings can be built through different combinations.

When designing the cutting file, based on the ordinary module unit, the jagged card slot is designed on the edge of the module to take into account the assembling problem of children. In this way, the building model can be assembled without glue and other additional tools so as to reduce the assembling difficulty of children (Fig. 7).

**Fig. 7.** Multiple block styles

#### **3.2 Virtual to Reality—Export Cutting Processing Files**

After completing the construction of the virtual model, the digital results are exported from the virtual world to the real world.

First, obtain the location information of the constructed module through unity3d, read a GridCell class mounted on each module, and record the row, col and level of the module. After each layer is built, the information of modules given a direction is recorded, and blocks corresponding to the direction are added to the array. In this way, when generating the DXF file, you only need to read the array of the corresponding direction, find the DXF file corresponding to the module name, use the netdxf library to insert each module separately as an insert block into the generated file, the starting point of the block has been the lower left corner as the origin coordinates. The corresponding three-dimensional model and DXF coordinate system data need to have a transformation formula, and transformation formula algorithm, as below.

$$\mathbf{x} = \mathbf{a} \ast \mathbf{col}, \mathbf{z} = \text{height} \ast \mathbf{Layer}, \mathbf{y} = \mathbf{b} \ast \mathbf{row} \tag{1}$$

where:

**x, y, z**: is the X-axis, Y-axis, and Z-axis coordinate position in the DXF file. **a**: is the constant of the unit length of the grid system. **b**: Is a constant of the unit width of the grid system. **Col**: indicates the number of columns of the block. **Layer**: number of layers of a block **row**: number of rows of a block.

#### **Algorithm1**

Identifying the Outer Contour of Assembled Blocks

1.Load the DXF file for each block.

2.Extract the polyline data from the DXF file for each block.

3.Store the polyline data for each block in a data structure.

4.Loop through each row, column and layer of the blocks.

5.Translate and rotate the polyline data based on the block's position and orientation.

6.Combine the polyline data for each block in the same row, column and layer to form the outer contour.

7.Repeat steps 4 to 6 for all rows, columns and layers to form the complete outer contour for the assembled blocks.

A file is generated for each elevation orientation. The corresponding files are saved in the project directory. After the generation of the cutting file and then opening with the CAD file, according to the location of the block, can separate the different planes of the block. Using the Outside Border Stroke Tool, draw the cut line with another centre line (Fig. 8).

# **4 Conclusion and Discussion**

The popularity of virtual reality technology has brought many possibilities to architectural education. Combined with the immersive feeling in virtual reality, it can increase the fun and authenticity in architecture and provide a more immersive experience environment. It breaks through the traditional teaching mode of staying in books and PPT and

**Fig. 8.** Generated module unit line draft and assembly model

inputs related knowledge of architectural construction into a VR virtual environment, which will greatly improve teaching efficiency.

If TV and mobile phones are a medium to know the world, VR virtual technology is a platform for children to experience, which can provide a more free and imaginative world. This operating system will be separated from the original virtual reality and reality, through the design of the two links, in line with the children's science education process. This building system can customize the design template, give more freedom to children in the design, and ensure that the built results can have enough richness and completion. To promote children's interest in building construction and a general understanding of the construction process.

Future improvement direction: Multiple children will be sought for the experiment. By analyzing the data on children's construction, we can make statistics on children's cognition of architecture and analyze the effect of virtual reality architectural education, as well as their preference and satisfaction with construction from the perspective of children (Fig. 9).

# **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **The Embodied Interaction with XR Metaverse Space Based on Pneumatic Actuated Structures**

Beibei Zang1, Tianjun Wang2, and Dan Luo3(B)

 Academy of Arts & Design, Tsinghua University, Beijing 10084, China Tisch School of the Arts, New York University, New York 10012, United States School of Architecture, University of Queensland, Brisbane, Australia d.luo@uq.edu.au

**Abstract.** This paper is based on our exploration of building an integrated-sensory XR interactive system breaking through the sensory boundary between physical environment and metaverse via pneumatic wearables. In short, our exploration mainly focuses on the following two aspects. Firstly, this research has adapted pneumatic-actuated wearable devices to satisfy the needs of clothing comfort and embodied interaction concurrently by exploring the downsized body-scale pneumatic system and programmable soft materials. Secondly, this research explores the possibility of using digital wearables as the linkage of physical body and XR metaverse to enrich the interaction between XR metaverse and physical environment, aiming at the real-time synchronization of physical wearer's and his virtual avatar in XR system.

**Keywords:** Embodied Interaction · Pneumatic Actuated Structures · XR Metaverse Space

# **1 Introduction**

Digitality has become ubiquitous, permeating every realm of our lives [1]. The boundaries of the human self may extend beyond the physical body, and the consciousness of those extended boundaries has been driven by the development of the outside world [2]. It's important for humans to adapt to the more profound sense of being humantechnological symbionts rather than the merely superficial sense of combining fresh and wires [3]. Since how humans interact with the world is greatly influenced and shaped by the tools used by them [3], wearables are considered as the second skin of people to inter-act directly and broadly with the built environment [4].

Pneumatic-actuated soft robotics has been a widely studied field recently with diverse applications [5]. Via pneumatic actuation, soft robotics has become increasingly accessible [6], and widely researched for application in biomimicry [7]. Furthermore, with the metaverse concept announced by Facebook, virtuality has been meant to have constant and seamless integration with existing physical reality [8]. Therefore, this research uses pneumatic actuated structures as smart garments to improve the comfort of digital wearables as the second skin of people. By using pneumatic actuated structures and physical sensors, this study will try to explore the digital wearables as the linker of physical body and XR metaverse, to enhance the XR sensory and physical comfort concurrently.

Contrasting with other digital bionic structures merely working with formal similarities. This study used pneumatic wearables as the representation of the latent living process embodied in creatures, like heart beating, blood flowing and breathing, to reconstruct the interaction between humans and their surrounding environments. For example, the pneumatic system combined the physical dynamics of plants and the flexible structures of human muscles. At the same time, the wearer's data and the signals from the surrounding environment can be transferred via wearables, triggering the interaction in both the physical and virtual system. In this way. A new dynamic of extended reality has been built with the wearables as an integrated interface of sensing and externalizing. Due to the bulkiness of the air pump of a traditional industrial pneumatic system, in this study, the pump and power system has been further downsized, so that it can meet the size restriction of the body-scale pneumatic system. Last but not least, with the synchronization of the pneumatic actuated wearables communicating via XR metaverse, the comfort of the physical body and the richness and vividness of the senses in the virtual world are simultaneously satisfied, providing prototypes verifying the feasibility for future application (Fig. 1).

**Fig. 1.** The embodied interaction with XR metaverse space on pneumatic actuated structures: metaverse space and pneumatic wearable

#### **2 Methodology and Prototype**

#### **2.1 XR Interaction Design—Physical Interaction of Digital Wearable**

Cognitive philosopher Andy Clark raised the idea in 1998 that human beings are best regarded as an extended system, a coupling of biological organisms and external resources [2]. His ideas coincide with the status of the metaverse today: more than 4.6 billion people can access the virtual worlds of the metaverse via smart phones, laptops, desktops, headsets or consoles [8]. However, communication with the virtual environment is based more on visual content, such as virtual reality platforms providing an engaging and immersive environment [9], or 3D virtual worlds for communication via PC and smart phones [10].

As researchers have concluded, the main feature of metaverse is a twofold link between the virtual and physical worlds: (a) behavior in the physical world influences the experience in the virtual one and, (b) behavior in the virtual world influences the experience in the real one [11]. Touching as a channel for a great variety of information has always been regarded as a crucial site for mediating social perceptions [12] and comprehensive environmental perception [13].

This project explores how the wearables interact with extended reality as an integrated interface of sensing physicality and externalizing XR experience. As a means of telecommunications, the combination of electronic components and sensors for data transferring between the human body and digital system has already been explored in the above-mentioned research. Therefore, instead of adding new applications of telecommunications into the field of human-computer interaction, this project is more about how to introduce the richness and vividness of the XR senses and physical wearable experiences.

In this research, digital wearable devices break through the sensory boundary between physical wearers and its digital avatars. The body movement and heartbeat change of the physical wearer are collected via three-axis acceleration sensor, heartbeat sensor and infrared distance sensor. The movement of the physical wearer is synchronized with its digital avatar, as the touch between digital avatars will cause haptic pressure changes of the physical wearer via its digital pneumatic wearable device in further application.

At the same time, pneumatic wearables visualize and externalize the latent life process of the physical wearer. The digital wearables' inflation and deflation according to the wearer's heartbeat rhythm, trigger changes in the bionic shape of the wearables and thereby change the tactile sensation of the wearer. Wearable devices draw on the techniques of three-dimensional tailoring and the study of muscle composition. Through the tailoring of pneumatics and clothing, the wearer's muscle dynamics during walking can also be creatively represented via wearables in both physical and XR worlds (Fig. 2).

**Fig. 2.** Interaction flow

In the current research of this project, only movement and heartbeat have been introduced into XR interaction and biotic visualization. In future applications, more scenarios of extended reality can be provided by richer collection of physical data such as environmental data of the wearer.

#### **2.2 Pneumatic Actuated Structures and Soft Material**

Under the spread of the digital environment and the popularity of digital devices, the fusion of traditional design and interactive technology has been accelerated in various fields [14], such as the application of 3D print in fashion design pioneered by Iris Van Herpan [15].

However, the above mentioned applications of digital technology devices have downplayed the typical characteristics of traditional fields, such as the basic requirements for comfort in the field of clothing [16]. Therefore, in this research, we have adapted the application of pneumatic systems and programmable materials to meet the basic needs of wearables for comfort and convenience.

#### **2.2.1 Programmable Soft Materials**

The heavy and expensive fabrication of the rigid body [5] hampers its efficiency and flexibility in body-device fit [17]. Therefore, silicone as a thoroughly explored material for soft robotics [18] has been used in this project as a programmable soft material. By computational design supported by Rhino and Grasshopper, we made a variety of 3D molds for silicone casting and fabricated the silicone airbags precisely according to their computed structures. Furthermore, through the combination of programable fabrication and pneumatic system, the deformation of functional garments can be assembled precisely to fit different modes of human activities. Last but not least, TPU has been fabricated as an alternative soft material for creating heat sealable sheets and laminated airtight layer [19]. Via patten prototyping designed in CAD and fabricated with laser cutting, the TPU airbag has also become programmable in the generating process from material to structure (Figs. 3 and 4).

#### **2.2.2 Pneumatic System**

The actuation system consists of three parts: Arduino toolkits for telecommunications between physical and digital space, sensors embedded in wearables, and pneumatic system for controlling the deformation of wearables. This section will introduce the downsized body-scale pneumatic system.

Air pump is controlled by two sets of solenoid valves which have three states (Inflate, Deflate and Hold), to meet the deformation requirement of the wearables via the inflation and deflation of airbags. For safety consideration, the solenoid valve selects the model SMC's S070C-SDG-32 powered by low voltage DC connecting with battery box.

Due to the bulkiness of the air pump of a traditional industrial pneumatic system, this project refits the air port of the portable air pump and connects it to the corresponding set of solenoid valves. Therefore, the entire pneumatic system only needs one portable air pump, two sets miniature electromagnetic valves and an Arduino toolkit connected with sensors. Through the design of digital wearable, the pneumatic system is organized into a pocket-sized box, which satisfies the portability and aesthetics of wearables simultaneously (Figs. 5 and 6).

**Fig. 3.** Programable Soft Materials: TPU and silicone and the process of their computed structures supported by Rhino and CAD. The Fig. 3 shows the process of TPU computed structures which was supported by Tongji University College of Architecture and Urban Planning and developed in DigitalFUTURES Shanghai 2018 Workshop

**Fig. 4.** Programable Soft Materials: TPU and silicone and the process of their computed structures supported by Rhino and CAD. The Fig. 3 shows the process of TPU computed structures which was supported by Tongji University College of Architecture and Urban Planning and developed in DigitalFUTURES Shanghai 2018 Workshop

#### **2.2.3 Morphology and Pneumatic Structures**

The combination of biological structure and morphology design has been widely used in various industrial fields, such as the mechanical properties of natural fiber cells [20] and soft elastic tissues of the human body [21].

To satisfy the comfort of the physical body and the vividness of tactile senses simultaneously, the morphology design draws on the dynamics of flexible structures and soft tissues, specifically, muscles as active elastic elements, skin as passive elastic elements and flytrap as pneumatic actuated structures.

The main structure of wearables follows the dynamic of muscle movement, while the holistic structure design takes both the static parts and kinematic joints into consideration.

**Fig. 5.** Pneumatic system design. The pictures on the left show how the pneumatic system is designed. The two pictures on the right show the testing of pneumatic system

**Fig. 6.** An example of pocket-sized pneumatic system with one portable air pump, one pair of two-sets miniature electromagnetic valves that can control two sets of airbags, and an Arduino toolkit which can connect sensors easily

Therefore, the pneumatic structure can be compatible with the daily activities of wearer, even assist the muscles' movement.

The airbag position mapping is generated by Grasshopper from the heat map of body movement. In the original prototype, the pneumatic structures consisted of airbag units fabricated with TPU and teflon after laser cutting. In further iterations, the pneumatic structures are added inflatable muscular structures made of silicone casting. In the composed pneumatic structures, the active parts fitted body movement are composed of muscular silicone structures, while the static parts are added on more biological features, to improve the comfort and richness of wearable devices (Figs. 7 and 8).

#### **2.3 Interaction with XR Metaverse Space**

The real-time synchronization between the physical wearer's behavior and one's virtual avatar is realized by Kinect motion capture via processing, physical sensors and their connections with metaverse space. As introduced in the physical interaction chapter, the wearer's activity data triggers deformation of the wearable device in physical space,

**Fig. 7.** Various prototypes of pneumatic air-bags' structures inspired by muscles and flytrap

and the mirrorly changes of the digital avatar's position and appearance happen concurrently via the linkage of physical data and metaverse space. Furthermore, beyond mere synchronization of the physical wearer's activities, a richer interactive experience has been introduced into metaverse space via HoudiniFX space rendering. Walking and turning of the physical wearer will trigger smoke and light effects in metaverse space. Therefore, the wearer of Digital Wearable can experience extended-sensory interaction between XR metaverse and physical environment with integrated perception of tactile changes and visual effects (Figs. 9, 10, 11 and 12).

# **3 Conclusion and Discussion**

The vision of this research is to build an integrated-sensory XR interactive system breaking through the sensory boundary between physical environment and metaverse. The current prototype has achieved the three goals mentioned above:


**Fig. 8.** Various prototypes of pneumatic air-bags' structures inspired by muscles and flytrap

**Fig. 9.** XR design: the connections between the wearer, the digital wearable and the digital avatar

**Fig. 10.** Real-time synchronization between the physical and digital ones by Kinetic motion capture via Processing

**Fig. 11.** Various experiences of metaverse space made by HoudiniFX

**Fig. 12.** An example of The Embodied Interaction with XR Metaverse Space based on Pneumatic Actuated Structures

Future development will focus on improving the delicacy of sensory transmission and the richness of interactive activities between physicality and extended reality. Specifically:


**Acknowledgements.** This research was supported by Tongji University College of Architecture and Urban Planning and developed in DigitalFUTURES Shanghai 2018 Workshop with team members Shaoting Zeng, Heidi and Junwon Yoon. The authors would like to express great thanks to the instructors and teaching assistants of the workshop, specifically, Behnaz Farahi, Jifei Ou, and Yuqiong Lin. The authors would also like to extend grateful appreciation to MIT Media Lab Tangible Media Group, whose work laid the hardware foundation and motivated the authors to do this research. And special thanks to Mitsuru Isshiki for his help with building 3D characters and metaverse space.

# **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Construction of Recreation Behavior Simulation Model of Public Space in Urban Waterfront—Taking Huangpu River in Shanghai as an Example**

Chunxia Yang and Ming Zhan(B)

Tongji University, 1239 Siping Road, Shanghai, China 945581064@qq.com

**Abstract.** This study constructs a multi-agent behavior simulation model to explore the quantitative simulation method of waterfront public space. Taking 6 waterfront public space samples along the Huangpu River in Shanghai as research objects, this study first collects environmental data and pedestrian behavior data through field survey, and then analyzes and processes the data to obtain the Spatial Attraction Weight (SWA) that expresses the relationship between pedestrian behavior and spatial elements. Then, based on the Anylogic platform, the pedestrian agent particles expressing people's characteristics are placed into the simulation environment based on the social force model. They interact in real time to dynamically simulate the pedestrian's behavior. Finally, fitting verification of the preliminary model is carried out. The qualitative comparison and quantitative correlation analysis are combined to enhance the accuracy. The behavior simulation model of waterfront public space built in the study can more realistically represent the pedestrian's behavior. It can realize the scientific prediction of the future use of waterfront space and provide more detailed reference for problem diagnosis and optimization.

**Keywords:** waterfront public space · Recreation behavior · Micro-behavior simulation · Multi-agent

# **1 Research Background**

As a unique space resource, urban waterfront has become an important recreational place in the city because of its continuous, open and water-friendly space characteristics. However, some extensive development and construction have made the urban waterfront public space low in vitality. The current problems are mainly manifested in improper connection of the base plane, single shoreline forms, limited public buildings, and dislocation of supporting facilities. The traditional small-sample survey, empirical design and pre-judgment make the allocation of spatial elements in waterfront areas not fully fit the user behavior preferences caused by waterfront characteristics. It is necessary to find more scientific, quantitative and intelligent analysis and simulation methods to guide the construction of urban waterfront space.

Multi-agent behavior simulation technology is a cutting-edge means of microbehavior simulation. It considers the interaction between individuals and the environment, and is suitable for simulating individual behavior. It can realize full-time dynamic simulation, multi-plan comparison and virtual situation preview. Its full-time interactive dynamic simulation can better present the self-organized behavior of the pedestrian under the influence of complex environmental space elements in the urban waterfront, and the output simulation results can be used as the basis for evaluating the future use of public space.

# **2 Research Object and Research Path**

Based on the Anylogic platform, this study constructs a behavior simulation model to simulate pedestrians' behavior in urban waterfron public spacet. It intuitively and dynamically presents the actual use of waterfront public space, and provides reference for the design, decision-making and management of waterfront public space. According to the principles of convenient accessibility, rich space, and open all day, this study selects six waterfront public space samples which are adjacent to each other in 3 pairs of the Huangpu River in Shanghai as objects for simulation. Six typical time periods (7: 00–9: 00, 9: 00–11: 00, 11: 00–13: 00, 13: 00–15: 00,15: 00–17: 00 and 17: 00–19: 00) are selected for the survey (Fig. 1).

**Fig. 1.** Six sample sites along the Huangpu River

The construction process of the behavior simulation model based on Anylogic platform includes four parts: basic data collection, data analysis and processing, simulation model construction and operation fitting adjustment (Fig. 2).

**Fig. 2.** Research framework

# **3 Research Key Links**

Behavior simulation technology was initially widely used in the simulation of evacuation behavior in indoor spaces such as stations, airports, gymnasiums, etc. In recent years, it has been extended to the simulation of outdoor pedestrian spaces and commercial streets. Currently, most of the relevant studies are rarely applied to the simulation of outdoor public space and daily recreational behavior. In order to carry out targeted simulation research, the following three key links should be defined for the model construction of urban waterfront public space.

#### **How to obtain and quantify the impact mechanism of the interaction between waterfront public space and pedestrian behaviors?**

Firstly, types and characteristics of spacial elements should be analyzed through investigation. Then the classification of waterfront recreational behaviors and their basic characteristics, including pedestrian movement, vision, environmental response and other characteristics, are studied. On this basis, we should explore the impact of waterfront spacial elements on pedestrian behavior and consider how to convert them into impact weight that can be recognized by the model.

This study focuses on the pedestrian behaviors and distribution in a specific waterfront space environment, and uses the spatial element attractiveness weight (SWA), which is an important model construction parameter to quantify the relationship between spacial elements and pedestrian behaviors.

#### **How to combine the advantages of agent model and social force model to build a behavior simulation model?**

Commonly used micro-behavior simulation models include cellular automata model, social force model, agent model magnetic force model, etc. Most of the current research is based on existing models to simulate behaviors with obvious regularities in various spaces. In order to improve the precision of the simulation, some scholars have improved the existing models. For example, Chiung-Hui (2011) improved the agent model based on the theory of visual attention, and used the agent program to represent the shopping behavior rules. Based on the nonlinear exit allocation strategy, Song et al. (2018) improved the social force model to avoid problems such as hitting a wall and distortion of exit capacity in simulation. However, current research rarely combines the advantages of each micro-behavior simulation model to construct a combined model and conduct targeted research. Meanwhile, research on the behavior simulation of complex urban public space lacks a mature combination model construction method.

In this study, the Anylogic platform that supports multi-method combination modeling is selected to build a combination model based on the advantages of the agent model and the social force model. The pedestrian path decision is dominated by the agent particles, which are placed in the simulation environment constructed based on the social force model, and the pedestrian path selection is expressed through the pedestrian behavior chain.

#### **How to conduct fitting analysis and verify the validity of the model?**

Fitting analysis is the key to verify whether the behavior simulation model can truly reflect the actual condition. The fitting method often uses an intuitive comparison between the measured and simulated results, and whether the fitting is effective depends on the empirical observation of the researcher. In recent years, quantitative fitting has gradually received attention, but more attention has been paid to the overall space, and there is a lack of fitting methods for local space. For example, Guo et al. (2014) applied Depthmap and SPSS correlation analysis software to fit the pedestrian behaviors in shopping centers under different spatial organization models. The model predicted the distribution of people flows in the overall space well, but the prediction accuracy of the distribution of local people flow in each layer did not meet expectations.

This study combines qualitative graphical comparison with quantitative correlation analysis, and takes the measured and simulated the number of people attracted by each attraction as the fitting basis to carry out sub-item simulation and fitting by time period and behavior type to improve the accuracy of the model.

# **4 Behavior Simulation Model Construction Process**

# **4.1 Basic Data Collection**

Basic data collection can be divided into environmental element data collection and pedestrian behavior data collection. This study classifies the diverse spatial elements of the waterfront public space into four categories of base plane, shoreline form, buildings, facilities and their subcategories. Using the method of field survey to collect the information of spatial elements. The record of the location and quantity of micro-spatial elements is especially paid attention. Pedestrian behaviors are divided into five categories of viewing, leisure, sports, entertainment, consumption and their subcategories. Pedestrians' waterfront recreation behaviors are diverse and random, with individual differences and obvious time differences. By means of mapping, counting and questionnaire, the information of pedestrian characteristics, behavior types, behavior locations, and behavior duration are obtained.

# **4.2 Data Analysis and Processing**

Compared with commercial streets or other urban public spaces, the urban waterfront public space has more complex and diverse spatial elements. Pedestrian behavior is more diverse and random due to the comprehensive influence of various environmental spacial elements. The distribution of pedestrian in waterfront shows that different spacial elements have distinct attraction to various pedestrian behaviors, that is, the probability of pedestrian choosing different spacial elements to carry out behaviors is different. The attraction weight of each space element expresses the relationship between spatial elements and pedestrian behaviors, which is an interactive parameter that can be identified by the model platform. This study adopts the Spatial Attraction Weight (SWA) in the waterfront public space to quantitatively express the regular relationship between the spatial elements and pedestrian behaviors in the waterfront public space, to establish the relationship between waterfront environment, pedestrian behaviors and the simulation model parameters. The calculation formula is as follows:

$$\text{SWA}\_{\text{(xy)}} = \frac{N\_{\text{(SA)}}}{N\_{\text{(T)}}}$$

N(SA) in x–y period, the total number of people of certain recreation behaviors in certain spatial elements.

N(T) in x–y period, the total number of people of all recreation behaviors in the waterfront public space.

#### **4.3 Simulation Model Construction**

Based on the Anylogic platform, the multi-agent behavior simulation model is constructed for six samples. Its modeling process can be divided into four parts: model operation process design, simulation environment modeling, pedestrian agent particles modeling and pedestrian behavior process modeling. Next, taking DongchangWaterfront as an example, the model construction process is introduced in detail.

#### **4.3.1 Model Running Process Design**

The process of pedestrian recreation behavior in waterfront public space can be understood as a kind of space-time change, with the dual attributes of time and space. The waterfront recreation process of pedestrians has no relatively unified model, and their behaviors in the waterfront public space are full of randomness and diversity. Recreational behavior is not only a movement in space. In the process of being attracted to stay in some areas, pedestrians are actually engaged in recreational behavior although they do not move in space, which needs to be expressed through time changes.

The model operation process is divided into five parts: particle emission, direction determination, particle susceptibility determination, particle attraction determination and recreation end determination.

#### **4.3.2 Simulation Environment Modeling**

Firstly, the CAD base map is imported into the Anylogic platform, and various spatial elements of base level, shoreline form, buildings, facilities and their subcategories in the site are translated based on the spatial markup module (Space Markupin) of the Anylogic to build a simulation environment. As static agents, the spatial module has three basic attributes: service radius, pedestrian capacity and attraction weight, which are defined through the Parameter module of the Anylogic agent library (Fig. 3).

**Fig. 3.** Simulation environment of Dongchang waterfront

# **4.3.3 Pedestrian Agent Particle Modeling**

In order to fully consider the characteristic differences of people, each pedestrian is simulated as a unique agent particle with its own attributes. Besides the three initial attributes of gender, age and behavior type, the pedestrian agent has four basic attributes: vision range, planned recreation time, basic speed and element perception radius. Pedestrian particles with different gender and age are given different basic attributes. In the Anylogic, the Parameter of agent library is used to express the four fixed basic attributes. Variable of agent library is used to express the dynamic attributes of pedestrian agent, such as moving speed, moving direction and moving time, which may change in the running of the model.

### **4.3.4 Pedestrian Behavior Process Modeling**

Pedestrians' behavior process corresponds to the model operation process. Based on the behavior module of Anylogic, a pedestrian behavior chain that presents the behavior characteristics of random selection of pedestrians along the waterfront is constructed to express the path selection and decision-making process of pedestrians in the waterfront (Fig. 4).

**Fig. 4.** Behavior chain

The pedestrian particles are put into the simulation environment built by the spatial markup module that based on the social force model. At each step of the continuous process, pedestrian agent particles can comprehensively simulate various attractions and repulsion forces in the space and make autonomous behavior decisions. It dynamically presents the behavior process of pedestrian waterfront recreation and the spatial distribution of pedestrians.

# **4.4 Fitting Analysis and Validity Verification**

On the basis of completing the preliminary construction of the combination model, the sub-item scenario simulation is carried out, and the results of the field survey are fitted by time period and behavior type.

# **4.4.1 Time-Period Fitting Analysis**

In this study, the preliminary model simulation results are compared with the survey and measurement results, and Spatial Attraction Weight (SWA) of each element is adjusted through multiple simulations to make the simulation results highly consistent with the actual situation. The fitting method is the same for each time period, and the final result basically conforms to the actual situation is obtained through fitting (Fig. 5).

**Fig. 5.** Summary of simulation results of Dongchang waterfront in different periods

On the basis of qualitative fitting, quantitative fitting analysis is added to improve the simulation accuracy of local space. SPSS Statistics is used to analyze the bivariate correlation between the simulated data and the measured data of the attraction points in each period.

There are a total of 62 spatial attraction points in four categories in Dongchang Waterfront. The actual number of people attracted in the site survey and the simulated number of people attracted by the 62 attraction points in each time period are counted. The bivariate correlation analysis of the measured data and the simulated data of each attraction point in each time period is carried out through SPSS Statistics. The Pearson coefficient is the minimum of 0.840 (7:00–9:00) in the correlation analysis results. The maximum is 0.989 (17:00–19:00), which is greater than 0.6. And the significance level is 0.000, far less than 0.01, which proves that the model is effective (Table 1).

# **4.4.2 Behavior Type Fitting Analysis**

On the basis of fitting analysis in different periods, the fitting of different types of behaviors is added. The model built in this study can simulate particles of a certain


**Table 1.** Correlation analysis of the number of people attracted by attraction points

type of behavior separately. We choose the 15:00–17:00 period with the largest number of people to compare the simulation results from the field survey results of each type of behavior. Taking the viewing behaviors as an example, the simulation results show that the viewing behaviors mainly take place at the wooden path, the rest platform and the cruise ship wharf on the concave shoreline, which is basically consistent with the measured results (Fig. 6).

**Fig. 6.** Partial enlarged figure of viewing behavior particle simulation result

Similar to the time-period fitting, the actual and the simulated number of people attracted for different types of behaviors at 62 attraction points in Dongchang Waterfront during this period are counted and analyzed. The results show that the Pearson coefficient was 0.703 for viewing behaviors and 0.986 for leisure behaviors, both of which are greater than 0.6. The significance level is 0.000, which proves that the model is effective.

#### **4.4.3 Model Validation**

The adjacent Dongchang Waterfront, which is basically the same as the surrounding environment of Dongchang Waterfront, is selected to verify the effectiveness of the model. The Spatial Attraction Weight (SWA) of the spatial elements after fitting and adjustment is given to the spatial elements in the adjacent waterfront to verify model validation. The simulation results are highly correlated with the measured results, which prove that the model is effective. The Spatial Attraction Weight (SWA) obtained by simulation can provide reference for the simulation of other similar waterfront.

# **5 Model Application**

Applying the above modeling methods, this study builds a behavioral simulation model for the waterfront section of the North Bund of Shanghai and carries out sub-scenario simulation. Firstly, the public space problem is diagnosed through simulation and optimization and improvement measures are proposed. Then this study improves the spatial elements such as various base planes, shorelines, buildings and facilities, and carries out the combination rehearsal and comparison, and puts forward the best recommendations with different orientations. Finally, an approximate combination is selected and compared with the reconstructed condition of the waterfront section of the North Bund which verifies the feasibility of the behavior simulation modeling analysis method (Fig. 7).

**Fig. 7.** Actually measured after transformation and pedestrian distribution of optimized and rehearsed before transformation

# **6 Conclusion**

This research innovatively applies behavior simulation technology to the study of recreational behavior in urban waterfront public space, which extends the application scope of behavior simulation. The behavior simulation model constructed in this study can perform multi-scenario rehearsal for the allocation scheme of spacial elements in the urban waterfront, and realize scientific prediction of future use. It provides a more detailed design basis for diagnosis and optimization of waterfront. The scene preview of the optimization scheme can avoid inefficient repeated renewal to a certain extent.

# **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Estimating the Impacts of Seasonal Variations of Streetscape on Dockless Bike Sharing Trip with Street View Images and Computer Vision**

Yongqin Zhao1, Sijia Wang1, Difan Chen2, Kaijie Huang3, Siyuan Zhang4, Waishan Qiu5, and Wenjing Li6(B)

 Independent, Shanghai, China Independent, Missouri City, CA, USA The New School, New York City, USA Huazhong University of Science and Technology, Wuhan, China Cornell University, New York City, USA The University of Tokyo, Tokyo, Japan liwenjing@csis.u-tokyo.ac.jp

**Abstract.** A significant portion of the cycling experience is influenced by the streetscape, and this impact varies throughout the year. The temporal dynamic of streetscape has been neglected in most previous studies, including urban public mobility route choices. This paper examines the correlation between dockless bike sharing and streetscape as well as spatial elements in different seasons using a large amount of GPS bike trajectory data collected by LIME. The study shows that: (1) DBS volume is significantly influenced by seasonal streetscape factors such as roads, cars, sidewalks, tree, and vegetation color; (2) How significantly these seasonal factors affect DBS volume differs in summer and autumn; (3) In both summer and autumn models, non-seasonal factors like mixed land use score, street network connectivity, etc., are significant. Some non-seasonal factors only impact the DBS volume in one season; (4) When adding subjective perception to models of both seasons, model explanatory does get improved very slightly.

**Keywords:** Seasonal variation · Dockless bike sharing · Street view image · Computer vision · Built environment

# **1 Introduction**

Bikeshare promotes sustainable travel, health benefits, and economic growth (Qiu and Chang 2021). Dockless bikeshare (DBS), compared to the docked bikeshare system, is getting more popular in the last decade due to benefits like accessibility and convenience (Gu et al. 2019).

There are observed research gaps in DBS seasonal study: (1) the development of mobile applications and cashless mobile payment have make bike sharing usage even more prevalent (Guo et al. 2022). However, as a new mode of transportation, DBS has received less attention than docked bike sharing (Guo et al. 2022). (2) A majority of precedent studies of DBS focus mostly on where a trip starts and ends, rather than the cycling experience itself. (3) limited examination of how seasonal streetscape affects cycling. Although a small number of studies consider temporal scale, yearly comparisons (Li 2021) offer limited help in comparing seasons, and studies addressing the association between seasonal climate and bike sharing didn't examine other seasonal environmental characteristics(Li and Kamargianni 2017).

The study (1) provides a quantitative study of DBS focusing on perceived environmental elements along the journey, (2) integrates SVI and CV to estimate how seasonality of street built environment impacts DBS usage at a fine spatial scale. (3) considers previously ignored seasonal environmental features like vegetation color.

# **2 Data and Methodology**

### **2.1 Study Area and Methodology**

Our study area includes the Town of Ithaca and a few nearby neighborhoods (Fig. 1).

**Fig. 1.** Study area

Figure 2 illustrates the framework of this study: (1) Using GPS bike trajectory data from LIME, a DBS system in Ithaca USA, we computed Seasonal Weighted Rides (SWR) to capture the cycling volumes of road segments in summer and autumn. (2) We collected SVIs in summer and autumn with Google SVI API. (3) We used PSPNet to compute the view ratio indices of streetscape elements, and used Mask R-CNN to count the number of streetscape objects. We also computed CV indicators (color deviation, L, A, B values in CIELAB color space) to present the seasonal color change of these three variables: tree, plant, and grass. (4) We quantified four subjective perception scores (accessibility, ecology, enclosure, scale) of street environment in summer and autumn with ML. (5) We collected and computed non-seasonal variables (typical POI, landmark POI, infrastructure, road type, land use mixed score, street network connectivity, terrain). (6) With OLS regression models, the seasonal environment attributes are comprehensively analyzed with their impacts on DBS volumes in summer and autumn.

**Fig. 2.** Research framework

#### **2.2 Data and Processing**

#### **2.2.1 Dependent Variable: DBS Data and Seasonal Weighted Rides (SWR)**

LIME provided us with the DBS trips data for this study, accessible through an API. LIME's app was used to collect the data. After the users agreed to LIME's terms and conditions for using the services, LIME recorded and analyzed their journeys. The dataset does not include identifiable individual information. To clean the data, trips that started or ended outside the Greater Ithaca area (Qiu and Chang 2021), trips with distances shorter than 0.05 miles or 264 feet (Qiu and Chang 2021), and trips with durations less than 3 min or more than 120 min were removed. The validated dataset has 102,178 trip records.

To make cyclists' choice and preference of routes comparable across different seasons, we have to capture the popularity of road segments in each season with Seasonal Weighted Rides (SWR). This conversion can be found in Eq. (1).

$$SWR\_j = SR\_j / (\sum\_{j=1}^{n} r\_j) \tag{1}$$

For segment *j*, *SRj* represents the volume of rides on it in this particular season, *n* represents the number of segments with rides on them in this particular season, and *<sup>n</sup> <sup>j</sup>*=<sup>1</sup> *rj* represents the total number of rides on all segments in Ithaca during this particular season. The *SWRj* that is aggregated to the sampling points*i* is the dependent variable of this study, and has the precision of street segments, so sampling points on the same segment would have the same value for one season.

To reduce the bias caused by data sparsity, we removed segments with a total annual volume of less than 500 riders (approximately 1.5 riders a day) from the aggregated data. Then we sampled points every 25 m in segments with lengths more than 50 m (which is too short). Therefore, 671 out of 10508 road segments were selected, and 2,082 sampling points were obtained from them.

#### **2.2.2 Independent Variable: Street View Measuring**

Google Street View (GSV) is only available in summer (Jun, July, Aug) and fall (Sept, Oct, Nov) in our study area, and out of 2,082 sampling points, there are only 1,170 points having GSV in both seasons. With GSV panorama ID and Street View Download 360 software (Street View Download 360, n.d.) we download the panorama in both summer and fall of each point. To extract the count of elements in a panorama SVI, we use a Mask-RCNN pre-trained on COCO dataset with ResNet-50 backbone. Then we use Pyramid Scene Parsing Network (PspNet) with pre-trained model psp\_resnet101\_ade to conduct image segmentation.

Undesired segmentation distortions might occur near the top and button of a panorama image, so we unwarp the panorama into images in 6 directions (Forward, Back, Left, Right, Up, and Down) with py360convert package and extracted the four directions at eye-level: forward(F), back(B), left(L), and right(R) (Fig. 3).

From percentage that the pixels of the specific visual element take-up of the total pixels of an SVI we calculated the visual ratio of an element in the image. Not all objective view indices will be input to the regression model after the Variance Inflation Factor (VIF) test, only 20 out of 28 visual ratio are kept, including: tree, road, grass, car, streetlight, wall, building, sidewalk, earth, water, plant, awning, van, person, bridge, railing, bicycle, minibike, ceiling, chair.

To study color and change of street greenness from urban cyclists' perspective, we used the CIELAB colorspace and extracted PSPNet pixels for three types of greenness. Converted from RGB to CIELAB using the Python-colormath library, we calculated average A and B values for each pixel and the standard deviation from actual values. To eliminate potential interference, we eliminated the L value (brightness) as SVIs are taken in different conditions.

We evaluate street perceptions using a 300-point pre-labeled dataset and an ML framework developed by related research (Qiu et al. 2023; Su et al. 2023) to evaluate the subjective perceptions score of the streetscape: Accessibility (accessibility to

**Fig. 3.** Panorama object detection with Mask\_RCNN, unwarping(in 4 directions: F, R, B, L), and semantic segmentation with PSPNet. Left: Summer SVI. Right: Autumn SVI

activities, attractions, and amenities), Ecology (detecting living organisms, animals, plants, humans, and their physical environment), Enclosure (the degree to which buildings, walls, trees, and other vertical elements define streets and public spaces visually), and Scale (human-sized and proportional elements). The dataset was collected from a crowdsourced visual survey of an expert panel and includes SVI input variables and 4 perceptual scores as output labels. 75% (225) of the dataset is used for training and 25% (75) for testing. Multiple ML algorithms are used, including K-Nearest Neighbors (KNN), Support Vector Machine (SVM), Random Forest (RF), Gaussian Process (GP), Gradient Boosting Regression (GB), ADA boost, and Bagging Regression. GP is chosen as the optimal model for predicting the target perceptions (Table 1).

#### **2.2.3 Independent Variable: Non-seasonal Variables**

We use Open Street Map (OSM) to get typical POI, infrastructure (transit facility), road types, and landmarks POI. The buffer zone radius is 500m for typical POIs, 100 m for infrastructure, and 1000 m (typical 5 min bike rides) and 3000 m (typical 15 min bike rides) for landmark POIs.

We use land use data collected from Tompkins County Open Data Portal (Land Use and Land Cover 2015; Tompkins County Open Data Portal, n.d.), and use a method


**Table 1.** Performance of GaussianProcessRegressor (GP) predictions

originally developed to calculate the evenness of distribution of the area of different use types (Frank et al. 2005). The calculation is shown in Eq. (2).

$$\text{LandueMix} = (-1) \ast \left(\sum\_{l=1}^{n} p\_l \ln(p\_l)\right) / \ln(n) \tag{2}$$

With *pi*, the proportion of land use *i* within the 500m buffer is divided by the total area within the 500 m buffer, while *n* represents the number of exclusive land use types within the 500 m buffer. This formula would calculate the mixed land use level within range 0–1, a higher value means a higher level of mixed land use.

We use Depthmap X to run the space syntax calculation. However, we finally choose Connectivity and Angular Integration (with segment length weighted, or SLW) after removing space syntax score with high multicollinearity (VIF > 10). Angular Integration (SLW) has two metric radius parameters: 250 and 1000 m. For a particular segment, Connectivity describes quantity of other segments it connects with, and Angular Integration (SLW) captures accessibility and how close(integration level) of this segment is to all others in terms of the sum of angular change(Angular Integration Space Syntax—Online Training Platform, n.d.).

We use high resolution(1 m) Digital Elevation Model (DEM) from United States Geological Survey (USGS) collected in May 2020. The medium value of slope within 5 m of sampling point along the road is chosen as the slope value.

#### **2.3 Model Architecture**

We start with a simple OLS model Eq. (3).

$$Y\_l = \alpha + \sum\_m X\_{(l,m)} \beta\_m + \epsilon\_l \tag{3}$$

*Yi* is the dependent variables of the sampling point. *X* is the independent variables that explains *SWR*. β is the coefficient of variable *m* that reveals how and to what extent variable *m* is related to *SWR*. Constant term α refers to the average *SWR* when all other variables are zero. Error term *<sup>i</sup>* captures elements that influence the *Y* but are not included in *X* .

A baseline model (M1\_Summer, M1\_Autumn) with significant variables was constructed using all variables except subjective perception. VIF was calculated in the whole process and only variables with VIF less than 10 are kept. Then all 4 perception scores are added (M2\_Summer and M2\_Autumn) (Table 2).


**Table 2.** Dependent variable, independent variable, and method of different models

# **3 Result and Discussion**

#### **3.1 Models Performance and Diagnosis Results**

Because DBS volume is not statistically significant with all variables, we remove unrelated variables from models. The remaining significant variables are used to build models after removing variables with high correlations and multicollinearity with a VIF test. The Regression diagnosis results show that: (1) All Autumn models have higher R2 than Summer models. (2) M1 model and M2 model have very close R2 with or without subjective perception. (3) There is high correlation and multicollinearity between the new perception scores and other segmentation results.

M1\_summer and M1\_autumn results show consistent significance for some variables in both seasons, such as visual ratios for roads, cars, sidewalks, grass color deviation, land use mix scores, and road network angular integration. Some variables are only significant in summer, like building and wall ratios and number of education POIs, while others like color deviation and lab\_A values of tree, lab\_A and lab\_B values of grass only show significance in autumn. For variables significant in both seasons, the positive or negative effects are consistent across seasons (Table 3).

#### **3.2 The Seasonal Variations of the Streetscape**

Cycling volume is positively impacted by the presence of roads and sidewalks, with dedicated bike lanes potentially available on a larger network. Studies of the built environment often examine both walking and cycling behaviors together (Mertens et al. 2016), so sidewalks play an important role as well. A pedestrian-friendly neighborhood promotes sustainable mobility and slows down traffic. Trees positively impact cycling in both seasons. Street greenery offers ecological benefits to neighborhoods, including providing shade for microclimate control, and creating an enjoyable environment for cyclists (Li et al. 2018). Waterfront areas provide a desirable setting for cycling, which aligns with previous research findings (Ding 2016; Lee et al. 2021; Song et al. 2021).

#### **3.3 Other Non-seasonal Variables**

Land use mix score is positively related to DBS volume in both season, aligning with prior evidence that the mix ratio of land use affects travel behavior (Van Dyck et al.


 **3.** Coefficients between bikeshare volume and selected variables

**Table**



(*continued*)


**Table 3.**

(*continued*)

#### 220 Y. Zhao et al.


**Table 3.** (*continued*)

(*continued*)



222 Y. Zhao et al.

2012; Kerr et al. 2016). Angular integration at 1000 m radius positively contributes to DBS volume in both seasons, in line with previous research suggesting that road network accessibility influences cycling behavior (Saghapour et al. 2017; Tucker and Manaugh 2018). However, angular integration at 250 m radius and connectivity are only significant in summer, which may be because autumn rides are more commuter-related. More research is needed to understand how space syntax impacts DBS at various scales and times. The number of points of interest (POI) is found to have a positive impact on DBS volume in both seasons. However, only educational POI has a significant positive impact in summer. This difference would need further research with more POI data.

#### **4 Conclusion**

Using GPS trajectory data, this study examines the correlation between DBS, streetscape, and spatial elements in different seasons. The study finds that: (1) seasonal streetscape factors such as roads, cars, sidewalks, tree, and vegetation color significantly influence DBS volume; (2) the significance varies in summer and autumn; (3) non-seasonal factors like mixed land use score, street network connectivity, etc., are significant in both seasons, some only show significance in one season; (4) adding subjective perception to both seasons improves explanatory slightly.

There are several limitations in this study that can be improved in future studies. Firstly, due to the data source limitation, only summer and autumn SVIs are collected in Ithaca. Finer temporal resolution can also be taken into consideration when SVI from more seasons or even months is available. Secondly, more advanced spatial model can be introduced to examine the spatial effect. Thirdly, microclimate-related data like temperature would better explain the seasonal variation.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Generation Scheme of IndoorGML Model Based on Building Information Model**

Ziyu Tong1(B) and Hang Zheng2

<sup>1</sup> Nanjing University, Hankou Road 22, Nanjing, China tzy@nju.edu.cn <sup>2</sup> Shanghai Municipal Engineering Degin Institute (Group) CO., LTD, Jiefang East Road 5,

Jinan, China

**Abstract.** In recent years, the concept of City Information Model (CIM) has received wide attention. However, the interior spaces are difficult to handle in CIM due to its complexity in terms of location and connection on 3D. Indoor Geography Markup Language (IndoorGML) is a data format standard for the exchange and representation of indoor space data and provided a method to describe interior space objects for CIM. However, the existing generation process is cumbersome and difficult to integrate semantic information. This study proposed a BIM-based IndoorGML model generation scheme. The scheme took the typical Revit model as the data base, extracted the location and attribute information of elements respectively, then generated the topologically expressed model integrated the semantic information. The study selected a hospital with complex interior spaces as a case study for the generation experiments of IndoorGML model. The result showed that the scheme is highly feasible even for such complex buildings. This study further calculated complex network-related attributes and analyzed the relationship between interior spaces to explore the application potential of the model.

**Keywords:** IndoorGML · BIM · Network analysis · CIM

# **1 Introduction**

In recent years, China's smart city construction has been developing rapidly. The corresponding concept of City Information Model (CIM) has also received wide attention since it was proposed. Generally, CIM integrates the concepts of BIM and GIS, and provides a basic platform of data management for the construction of digital twin cities. However, whether it is BIM or GIS, the data objects in its underlying logic are entities, such as buildings, plots, and roads at the city scale, and walls, doors, and windows at the building scale. The organization and management of space-based objects are lacking in all these data models. Among them, the interior spaces are more difficult to handle due to their complexity in terms of location and connectivity on 3D (Becker et al. 2009). The lack of spatial objects has a severe impact on the integrity and adaptability of CIM.

Indoor Geography Markup Language (IndoorGML) is a data format standard proposed by the Open Geospatial Consortium (OGC) for the exchange and representation of indoor spatial data (Kang and Li 2017). IndoorGML divides the building indoor spaces into a set of cellular spaces and expresses the connectivity between these spaces with Node-Relation Graph (NRG). Moreover, IndoorGML can add semantic and geometric information to the model by external linking references. IndoorGML has provided a method to describe interior space objects for CIM (Liu et al. 2017), and has been applied in emergency evacuation (Alattas et al. 2020) and interior navigation (Yang et al. 2021). However, IndoorGML is still at the early stage of development, and the existing generation process is cumbersome and difficult to integrate semantic information (Jeong et al. 2018; Srivastavaa et al. 2018).

According to the shortcomings of the existing generation methods of IndoorGML, this study proposed a BIM-based IndoorGML model generation scheme. The scheme took the typical Revit model as the data base, extracted the location and attribute information of model elements respectively, then generated the topologically expressed NRG model and the solid expressed building model, and finally established the mapping relationship by model element ID to integrate the semantic information. Since different operations were involved, the scheme used a combination of different platforms such as Dynamo and Grasshopper. The study selected a hospital with complex interior spaces as a case study for the generation experiments of IndoorGML model.

This study further calculated complex network-related attributes and analyzed the relationship between interior spaces to explore the application potential of the IndoorGML model. The study analyzed the accessibility of doctor and patient flows separately based on NRG in the IndoorGML model. The result showed that the generation scheme is highly feasible even for large buildings with very complex interior spaces.

# **2 Method**

The BIM-based IndoorGML model generation scheme can be divided into three modules: BIM information extraction, geometric model construction, and semantic information integration. Figure 1 presented the generation scheme.

#### **2.1 BIM Information Extraction**

The amount of data in the BIM model is relatively large and not all of the information in the model can be used to create the IndoorGML model, so it is important to identify information needed and extract it from the BIM model. This study uses the Dynamo plugin in Revit as a tool for extracting BIM information.

The IndoorGML model expresses the adjacency and connectivity of the interior spaces, including rooms, corridors, doors, stairs and lifts. These elements can be filtered through the Dynamo plug-in in the Revit model. For each element, the location information is directly exported as a spreadsheet file that would be used to generate the IndoorGML geometry model.

For the semantic information, since IndoorGML supports external references to Industry Foundation Classes (IFC) files, the BIM model can be exported as an IFC file for subsequent external references in IndoorGML. To reduce the amount of arithmetic

**Fig. 1.** The generation scheme of IndoorGML model based on BIM

involved in external referencing, the BIM model should be made lighter by removing unnecessary information.

#### **2.2 Geometric Model Construction**

The IndoorGML geometric model includes two parts, one is the topological representation, the NRG model, which corresponds to the *multilayeredGraph* module in the IndoorGML data, and the other is the solid constraint representation, i.e. the walls, floors and doors, which corresponds to the *primalSpaceFeatures* module in the IndoorGML data. Both are based on the element location information extracted from the BIM model and generated by the appropriate algorithms.

The NRG model consists of the *State* and *Transition*. The *State* is represented by points that indicate interior spatial units, while the *Transition* is represented by edges that indicate the connectivity between spatial units. The NRG model expresses only the topological relationships of the interior space without any semantic information. The *primalSpaceFeatures* can be divided into *CellSpace* and *CellSpaceBoundary*.

According to the composition of IndoorGML, it is possible to construct an IndoorGML model by converting the data information derived from the BIM model. However, in larger buildings, it would take a lot of time and effort to manually input the data one by one, so it is necessary to design mapping rules and algorithms to quickly generate IndoorGML standard text data from the extracted BIM data.

IndoorGML content is divided into four main sections, with the data in each section presented in a fixed format. This study uses Python programming to generate IndoorGML text. Taking the State section as an example, after reading the spreadsheet file, each row of data is exported as a piece of texts in IndoorGML text until all the State data has been exported. The output of the other three sections is similar to that of the State section. Once the individual sections of IndoorGML data have been created, the data is merged and the complete IndoorGML model is obtained.

### **2.3 Semantic Information Integration**

The external references in IndoorGML are mainly used for *CellSpace* and *CellSpace-Boundary*, which correspond to rooms and doors respectively in the BIM model, and to *IFCSpace* and *IFCDoor* respectively in the IFC. Considering that the purpose of the semantic information is to enrich the functional information of the cell spaces, the integration of the BIM room information is mainly carried out for the *CellSpace* part.

A complete BIM model exports a large amount of IFC files, so the required *IFCSpace* information needs to be filtered out. The key to referencing external files in IndoorGML is twofold: determining the location and filename of the referenced IFC file, and matching the *GUID* number of the referenced object in the IFC text with the ID of the object in IndoorGML. The first point is relatively simple, as long as the path name of the IFC file is entered correctly, while the second point requires further processing of the data is required.

In the exported IFC file, each *IFCSpace* has a unique *GUID* number and the room name. Each *CellSpace* in IndoorGML also has a unique ID number. In the previous step, the room name of each *CellSpace* is obtained through the Dynamo. The room name corresponds to the ID number of the *CellSpace*. Therefore, using the room name as a medium, the ID number of the *CellSpace*, the room name and the *GUID* of the *IFCSpace* can be related one-to-one to obtain the *GUID* corresponding to each *CellSpace*.

Since each *CellSpace* has one and only one reference object, the pathname of the IFC file and the *GUID* are input as separate variables to the corresponding location in the IndoorGML text to complete the external reference of the IndoorGML model to the IFC information. After completing the external reference, the IndoorGML model implements the integration of the semantic information.

# **3 Case Study**

Unlike small buildings such as office buildings, the interior space structure of a hospital is more complex, the process of generating the model is more computationally intensive, and the generated network model contains more information, which will effectively test the feasibility of the above IndoorGML generation scheme.

#### **3.1 Hospital BIM Model Data Processing**

The BIM model of a hospital was selected as the data source for IndoorGML model generation, and the interior space of the model is shown in Fig. 2. The BIM model was first processed to extract the data information needed to build the IndoorGML model, including information of rooms, corridors, doors, stairs, lifts, etc. The above information was extracted using the Dynamo plugin.

**Fig. 2.** The BIM model of the hospital

The exported room and corridor information contained room area, number of occupants, heating and cooling loads, etc. To generate the IndoorGML geometry model, the room contour lines were required, and to integrate the semantic information, the room name was required. The extracted door information includes the axis endpoint coordinates and the centre point coordinates, whereas for stairs and lifts only the centre point coordinates were extracted. All of the above information was saved as separate spreadsheet files.

#### **3.2 Hospital IndoorGML Model Generation**

To generate the NRG model, the data required included the contour line coordinates of the rooms and corridors and the axis and centre point coordinates of the doors. The room contour coordinates and door centroid coordinates were imported into Grasshopper and the geometric reconstruction of the room contours was completed. Afterwards, the distance from the centre point of each door to the contour line of each group of rooms was calculated, and the relationship between the doors and the rooms was determined. The centre point of the room contour line would be used as the *State* data in the NRG model, while the line from the centre point of the room to the centre point of the door would be used as the *Transition* data (Fig. 3).

**Fig. 3.** The NRG model of the hospital

The data processing for the vertical circulation spaces was similar to that for the room units. After connecting the centre point of the contour line of the stairwell or lift room to the centre point of the corresponding door, it was also necessary to create vertical connections to complete the construction of the NRG model.

After the generation of the *multiLayeredGraph* part was completed, the *primalSpace-Features* part of the model, i.e. *CellSpace* and *CellSpaceBoundary*, could also be generated in Grasshopper. Finally, the *multiLayeredGraph* and *primalSpaceFeatures* sections were merged to complete the IndoorGML geometric model. By mapping the *GUID* of the room in the IFC file and the ID of the *CellSpace* in the IndoorGML geometric model, the semantic information was integrated into the IndoorGML model. Figure 4 demonstrated the final model in the IndoorGML viewer.

**Fig. 4.** The IndoorGML model of the hospital displayed in the viewer

# **4 Semantic Network Analysis of the Hospital IndoorGML Model**

The semantic network emphasizes the adjacency of spaces to each other. Based on the information carried in the IndoorGML model, a further scientific and rational evaluation of the neighbourhood relationships in interior spaces can be performed through semantic network analysis.

#### **4.1 Semantic Network Model Construction**

In the IndoorGML, the NGR model is an abstract representation of the connectivity of interior space in the form of points and lines. The semantic network model differs from the NRG model in its composition. In spaces separated by doors, the NRG model is composed of cell-door-cell, whereas the network analysis is aimed at spatial cells. Therefore, the NRG model needs to be reconstructed. Figure 5 showed the cell-cell network model.

The model consists of a number of single straight lines whose endpoints are the centroid coordinates of two adjacent spatial units, and all spatial unit adjacencies can be obtained by node pairs. As each point corresponds to the room name of the spatial unit it represents, the coordinate of the point could be replaced by the room name derived from the semantic information of the IndoorGML model. The connection between all rooms is then obtained.

#### **4.2 Centrality and Modularity Analysis**

Centrality and modularity are two common metrics used in Graph Theory, where the former is used to determine the importance of nodes in a network, and the latter is

**Fig. 5.** Connections between the CellSpaces

used to identify the association structure of a network. By applying the two metrics to the semantic network analysis of indoor spaces, the weight of different spaces in the neighbourhood and the grouping of spaces with similar neighbourhoods can be obtained respectively.

Gephi is a complex network analysis software that allows to quickly build semantic networks and perform exploratory data analysis, link analysis, social network analysis, and many other types of network analysis. In this study, Gephi was used to calculate the centrality and modularity of the sematic network model of the hospital. The calculated results were fed back into the external reference model of the IndoorGML to improve the sematic information. Figure 6 presented the centrality and modularity calculations of the hospital.

The results of the centrality calculations showed that the core traffic space has the highest centrality among the functional spaces of the hospital, followed by the nurses' station and its associated waiting area, and then by hospital-specific functional spaces such as CT rooms and ultrasound rooms, while offices, duty rooms and tool rooms are less centrally located, and the least centrally located spaces are toilets, dressing rooms, medicine storage and equipment rooms.

The results of the modularity calculation divided the hospital into 17 groups, where spaces with similar functions and in the same functional division were mostly in the same

**Fig. 6.** The centrality (left) and the modularity (right) of the hospital

group, consistent with design experience. There were also some exceptionally small groups that were not functionally zoned properly. From a modularity perspective, there is potential for improvement in the design of these spaces in terms of their connectivity to the surrounding spaces.

#### **4.3 Flow Paths Analysis**

The design of flow paths is an important part of architectural design, especially in hospital design, including the separation of patients and doctors and the separation of cleaning and sewage. The flow paths determined the accessibility of the rooms. To facilitate the classification of the flow types of the paths in the NRG model, the properties of the different spaces were determined according to their function. The lines in the patient's accessible space were extracted to form the patient flow paths. The lines in the doctor's workspace were extracted to form the doctor's flow paths. In addition, the results of the corresponding network analysis could be assigned to the IndoorGML model.

Figure 7 presented the accessibility of the flow paths for the patients and doctors. The most accessible patient paths passed by the nurses' stations and the medical and technical areas, while the more accessible doctor paths passed by the medical and surgical areas. Such analysis and visualization could help to evaluate the interior spaces design from the perspective of different user groups.

# **5 Conclusion and Discussion**

Indoor spatial data is an important part of building CIM in the future, and constructing a digital model of indoor space can provide an effective management tool for spatial data. This study proposed a scheme for the generation of BIM-based IndoorGML models. And with the generated models, the analysis of spatial connectivity and accessibility in IndoorGML models was also explored.

**Fig. 7.** Accessibility of the patient paths (left) and the doctor paths (right)

Using a hospital as a case study, this study validated the effectiveness of the generative scheme. Despite involving the combined use of multiple software platforms, the final generated IndoorGML model fully represented the geometric information of the interior space and the corresponding attribute information. Moreover, through network analysis, the model also demonstrated the centrality and modularity of the hospital interior space, as well as the different flow of patients and doctors. The result showed that the generation scheme is highly feasible even for large buildings with very complex interior spaces.

This study is still at a preliminary stage. The use of multiple software in the operation to reduce programming work made the whole process still cumbersome. Simplification of the operational steps will be considered to avoid the involvement of multiple software. In addition, the IndoorGML model needs to be further explored with its application in interior spatial analysis as well as in CIM platforms.

#### **References**


Yang, J., Kang, Z., Zeng, L., Akwensi, P.H., Sester, M.: Semantics-guided reconstruction of indoor navigation elements from 3D colorized points. ISPRS J. Photogramm. Remote Sens. **173**, 238–261 (2021)

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Designing a Systematic Experiment to Investigate the Effect of Ambient Smell on Human Emotions in the Indoor Space; Introducing a Mixed-Method Approach**

Mohsen Kafaei1(B) , Jane Burry1, Mehrnoush Latifi1, and Joseph Ciorciari2

<sup>1</sup> Department of Architectural and Industrial Design, School of Design and Architecture, Swinburne University of Technology, Melbourne, Australia mkafaei@swin.edu.au

<sup>2</sup> Department of Psychological Sciences, School of Health Sciences, Swinburne University of Technology, Melbourne, Australia

**Abstract.** Studies have indicated that built environments affect all aspects of human life such as emotion, perception, behavior, health, and well-being (Cooper et al. 2011). Built environments are formed from the combination and juxtaposition of visible and invisible environmental variables. In recent years, common techniques such as virtual reality, augmented reality, digital twins, and artificial intelligence have enabled researchers in the field of architecture and urban design to simulate environmental conditions to investigate the impacts of environmental variables on humans. However, the studies conducted in this field of human comfort are mostly focused on the impact of environmental variables such as form, temperature, humidity, and sound, and in fewer studies, up-to-date methods and technologies have been used to simulate and investigate the impact of smell on humans. Most of the studies that have investigated the effect of ambient smell on humans, carried out in the discipline of architecture and urban design, have used traditional tools and methods (questionnaire, interview, observation) rather than advanced technology and tools drawing on neuroscientific knowledge and technique to measure the effectiveness of the ambient smell on human. They have used unmasked scents or real-world environments rather than being able to simulate environmental conditions. This article highlights the significance and necessity of employing simulation methods to investigate the impact of environmental smells on humans. Additionally, it presents the methodology of an experiment for studying the effect of indoor environment smells (with a case study of an office environment in the initial phases) on human emotions, utilizing a mixed-method approach. Analysis of some parts of the data from this experiment showed that exposure to the fragrance of the jasmine flower pleasant (flower) and the odor of the rotten orange peel (unpleasant) can cause changes in the electroencephalography (EEG) power across different bands among participants.

**Keyword:** Ambient smell · Built environment · Simulation methods · Electroencephalography · Neuroarchitecture · Physiological signals

#### **1 Introduction**

Our built environments comprise both visible and invisible environmental variables such as geometry, color, material, sound, temperature, and humidity, working together to create an atmosphere in our built environment that provokes certain perceptions or triggers certain emotions in individuals. So, in combination these environmental variables can affect our multi-sensory experience of space and have significant impacts on our emotions, behaviors, health, and well-being in a number of ways, both positively and negatively (Countryman and Jang 2006). Accordingly, it is essential to study and investigate how these variables can enhance the overall experience of a space. One of the invisible environmental variables in every space is "ambient smell". Smelling is part of the breathing process, and while breathing, a person willingly or unwittingly inhales the smells of the environment and the space around them, which can affect various aspects of a human's existence (Goel and Grasso 2004; Lehrner et al. 2005; Sowndhararajan and Kim 2016).

Many studies conducted in different disciplines have shown that smells can have a wide range of effects on different dimensions of human existence. According to literature the sense of smell has a special relationship with the processing of emotions due to the unique connection of the olfactory system to limbic system, and for this reason, triggers the recall of memories and emotions of experiences associated with a particular smell (Croy et al. 2011; Perkins and McLean 2020; Kadohisa 2013; Bowring 2006). Just as smells can create identity for individuals, they also have the ability to create identity for spaces in a positive or negative way, in such a way that a person can recognize the smell of an environment as acceptable or unacceptable in terms of identity (Maria 2016). Studies show that environmental fragrances can reduce the stress experienced by employees in workplaces. Ambient fragrances can increase aesthetic appreciation, optimize performance and increase creativity, as well as improve air quality as a remedy for multi-symptom health problems such as "sick building syndrome" (Damian and Damian 2006). It has also been found that smells can have a great impact on behavior and encourage or prevent individuals from doing something (Kuppens et al. 2017). For example, extensive research conducted in the discipline of marketing and business on investigating the impact of odor indicates that smell can change customer behaviors such as spending, gambling, purchasing desire, the length stay in the store, searching and choosing products, the desire to buy more (Lin et al. 2018).

#### **2 Neglect of Smell in Architecture**

Although many studies carried out in other disciplines show the role and impact of smells on different aspects of human experience, the parameter of ambient smell has received little attention in architecture and urban design and is mostly limited to removing unpleasant odors from indoor and urban spaces (Quercia et al. 2015; Barbara and Perliss 2006). It seems that the full potential of smell has not been utilized in a deliberate and strategic way (Bouchard 2021). It is perhaps because of this kind of viewpoint and thinking towards the olfactory dimension that studies investigating the impact of environmental odors are very limited, incomplete, and undeveloped, and in most cases, they still seek to investigate the impact of unpleasant odors on humans rather than investigating and the role of positive smells.

The environmental variable of smell has a complex, transitory, ambiguous, and unstable identity. Every smell is a complex mixture of hundreds of different volatile chemicals that change every moment, making it very difficult to record and capture (Margolies 2001; Henshaw and Cox 2009). So, the study of ambient scent is a complicated task due to invisibility of the parameter, the subjective nature of odor perception and individual's perception, the difficulties in quantifying and measuring odor intensity and composition, and the interaction of other sensory stimuli. These unpredictable and dynamic characteristics of smell make it difficult to develop comprehensive and robust methods for evaluating and controlling ambient scent in various settings.

In addition, the lack of a systematic methodology for carrying out research in architecture and urban design on the impact of smells on human experience has caused the studies conducted in this field to suffer from many shortcomings. Most of the studies (Henshaw 2014; Bouchard 2013; McLean 2012, 2013, 2017) carried out in this field are limited to identifying, categorizing and investigating the impact of smells in urban spaces in the form of smellscape, smell walking and smell mapping projects. Many studies lack standard and systematic empirical criteria and protocols, and the instruments and techniques used in these experiments are very limited (Balez 2002) and in most cases include methods and tools such as questionnaires, interviews, and observations, which have been traditionally and historically used in architecture (Quercia et al. 2015; Nanda et al. 2013). Given the complexities inherent in the field of ambient odor study, it is crucial to adopt innovative methods, tools, and techniques from other disciplines to be able to support designers to get better insight on the impacts of smells on human perception and be able to consider smell as part of design process. This necessitates a multidisciplinary research approach to integrate knowledge and techniques from a broad range of fields such as olfactory sciences, psychology, neuroscience, aromachology, and cognitive sciences, to be able to apply in architecture.

Based on this, we need to apply and use methods and techniques that can have the necessary capacity and potential for research and study in the field of investigating the impact of the smell of the environment on humans with multidisciplinary approaches. One of the solutions that can help to define, plan, and design studies and research in the field of environmental smell, in the form of systematic experiments with empirical criteria, is the use of methods and techniques of environmental simulation studies. Simulation is a process by which real world scenarios can be studied and analyzed by creating a virtual world or minimizing the scale of the real world (Mishra and Patnayaka 2015). Simulation generally refers to experimental modeling or representation of particular environments and events, which can include computer models, laboratory studies, role-playing and game analogs of social situations, scale models and designs related to architectural design projects (Maransa and Stokols 1993). Using the powerful environmental simulation techniques and tools in the discipline of architecture and urban planning can help designers to evaluate and analyze various environmental parameters and their impact on the environment user (Mishra and Patnayaka 2015).

#### **3 Simulation and Study of Ambient Smell**

In recent decades with the development of computer technologies and digital tools and various technologies such as virtual reality, augmented reality, advanced computing tools, digital twins, etc., new simulation techniques have been adopted by architects (Werner and Schindler 2004; Drettakis et al. 2007; Franz et al. 2005). The recent advancements in computer technology and digital tools, have enabled designers and researchers to integrate performance analysis into design processes and gain a deeper understanding of the impact of invisible environmental variables on design and human perception. However, despite these cutting-edge technologies, the field of ambient odor has not yet fully leveraged these tools and techniques to identify, categorize, and understand the behavior of ambient scents and their impacts on various dimensions of human life. Still less are these new techniques supporting designers to design with smell as a constituent environmental material.

To know which methods, tools and techniques related to simulation studies to use for ambient smell studies, it is necessary to first identify all the criteria, standards and protocols that can affect the research results from the point of view of different branches of science and then to choose appropriate methods and technologies to conduct the research. The studies conducted on environmental odors in the discipline of architecture and urban planning are very few and in many cases this research does not employ standard protocols and criteria. Therefore, the variables affecting these studies have not been accurately identified and categorized. It is thus necessary to investigate and study a wide disciplinary range of research on environmental odors and their effects on humans, to identify the variables that can affect the experiment method and results, and based on that, design experiments.

For example, studies in the discipline of aromachology that investigate the effect of odors on human mood, physiology, and behavior have specified criteria for conducting experiments in this field, which include: clarity of test objectives and hypotheses, measurement of odors using appropriate laboratory methods, appropriate number of participants and control groups, data analysis using appropriate statistical methods, as well as review of data and results by scientific peers and publishing them in reputable scientific journals (Herz 2009). Also, studies conducted in the disciplines of olfactory science and psychology have shown that not only the intensity of a smell in a scale of high, low or medium can change a person's perception of that smell and the environment and affect the test results, but the strength of the smell and the level of perception and individual's sensitivity to smells can greatly affect the test results (Maggioni et al. 2020). Therefore, to conduct simulated experiments, it is necessary to consider criteria such as odor intensity, concentration, evaporation rate and stability. In addition to the characteristics of odor such as intensity of smell and chemical compositions, a person's biological and personal characteristics can affect how individuals perceive and experience smell.

Regarding the role of data collection tools and techniques, Lin et al. (2018) state that the review of 20 studies that examined the effect of smell on human emotions, both through laboratory experiments and field experiments using measurement tools, revealed that self-reporting showed no significant difference with only marginal statistical divergence between the two methods. This shows that in addition to using subjective methods, such as self-reporting tools, it is necessary to use objective tools and techniques to collect data. In many cases, for various reasons, the participant cannot express and describe her/his mood and emotions towards a stimulus (Jatupaiboon et al. 2013). So, recording and analyzing brain and peripheral signals makes it possible to understand the internal aspects of human emotions (Khalili and Moradi 2009). In addition to the important criteria and parameters from other disciplines, it was also necessary to consider the parameters and criteria within the discipline of architecture and urban planning for simulation studies of the impact of environmental odor.

# **4 Designing an Experiment to Study Smell Using a Mixed-Method Approach**

This research proposes a mixed method to investigate the effect of ambient smell (pleasant and unpleasant) on human emotions in the indoor environment (office) using simulation techniques and qualitative and quantitative data collection tools. A series of experiments was designed and conducted based on a comprehensive literature review and the collaboration of a team consisting of experts in architecture, urban design, and psychology. The parameters affecting the smell of the environment and its perception by humans were identified and categorized.

In order to achieve a rigorous framework, pilot studies were carried out involving 14 participants. These pilot studies were conducted to evaluate the test method and data collection tools (which were developed based on a thorough literature review contextualized in the field of architecture). The findings from the pilot studies are used to refine and, if needed, modify the research method. This experiment is designed by considering the criteria and protocols of different disciplines for research on smell and considering the influential parameters of architecture in this field including:


the environment. Therefore, the indices and the characteristics of odors in the environment such as concentration, intensity, persistence of odors, and rate of evaporation of odors must be taken into account in simulation experiments.

– Type of sniffing: Naturally, when a person is in a given environment, (s)he does not smell the atmosphere actively (intentionally and consciously), but passively (unintentionally and unconsciously) smells and perceives the odor of the environment along with the air she/he inhales. The knowledge and insight on the type of sniffing can inform design of experiments, and particularly to avoid potential impacts from participants consciously focusing on their sense of smell during the tests. Humans unconsciously detect scents in their environment in various contexts without actively seeking them out. Therefore, it is important to keep the objectives of relevant experiment undisclosed to the participants. Accordingly, the objective of the study is kept undisclosed during recruitment and before conducting the experiments to prevent conscious detection of the scent by participants. Moreover, as a crucial strategy to prevent visual identification of the scent sources by participants, the source of the scent used in the experiment is concealed and visually unidentifiable in the test rooms.

### **4.1 Method and Procedure**

In this experiment, two test rooms are considered, in one room the smell of jasmine is emitted and in the other the odor of rotten orange peel is spread. At first, the participant is placed in the preparation room, and the explanations required about the test are delivered to them and they sign the consent form. Then, the biosensors are worn to measure Physiological signals and the participant enters the first test room. The participants are asked to sit on a chair in the test room for 3 min without any movement and then perform a task and complete the questionnaire. After a few seconds of sitting on a chair in the room, it is expected that the smell of the environment can be perceived and smelled by the participant. The psychological task performed by the participant simulates a personal office activity adding to the environmental simulation of an office. This is also an important diversionary tactic that makes participants think they are being tested on the puzzle they are doing and so never consider smell. Also, the analysis of the results of this task is an important criterion for whether the smell affects their cognition by recording the speed and efficiency of completing the task or not. Throughout the whole period of the participants' presence in the test room, their brain and peripheral signals are recorded by biosensors. Then, the person leaves the first test room, sits in the waiting room for 3 min until the effect of the smell of the first room is removed (washout period) and then enters the second test room, and all the steps are repeated again. All test steps are summarized in Fig. 1.

#### **4.1.1 Setting (Controlled Experimental Conditions)**

To perform this experiment, three rooms (two test rooms and one preparation room) were prepared, all of which have the same visible environmental parameters, including form, geometry, proportions and color. Each testing room is 4.30 m<sup>2</sup> with identical visual and insulation conditions. The interior environment is considered to be a one-person office in terms of functionality. The furniture inside the test rooms includes a desk, a chair, a

**Fig. 1.** Test steps

drawer (To hide the source of smell) and a device for recording environmental variables (see Fig. 2). The studies that have investigated the cross-modal perception of the sense of smell with other senses such as auditory and visual show that this interaction between different senses can have an effect on human perception of the environment and their emotions (Morrison et al. 2011; Mattila and Wirtz 2001; Gottfried and Dolan 2003). For this very reason, the color of the test rooms was considered white to reduce the effect of the visual sense on the sense of smell. In addition, it is necessary to measure and monitor the environmental variables to ensure that the values of each of these variables are the same during the experiment and for all participants. The research conducted by Nimmermark and Gustafsson (2005) shows that the emission of odor is strongly related to the water vapor pressure and the control of temperature and humidity may reduce the concentration and emission of odor. So, microclimatic changes of the environment including temperature, humidity, air quality (CO2 level), and noise are recorded by the Netatmo (NHC-P2) Smart indoor air quality device to ensure that environmental conditions of the experiment are identical and stable in both rooms. The temperature of the rooms is 20–22 °C, the humidity is 45–55%, the air quality is 400–500 ppm, and the noise is 40–45 db.

**Fig. 2.** A, Photo of experiment room. B, Floor plan indicating the position of participant and the items in the experiment rooms

### **4.1.2 Ambient Smells**

In one room, the smell of jasmine flower (pleasant) is spread from an invisible and unannounced source and in another room, smell of rotten orange peel (unpleasant) is similarly spread and can be smelled by the participants. The smell source is hidden in the drawer and at a distance of 90 cm behind the participant's head. It is necessary to calculate the intensity and concentration of odors as well as the rate of their evaporation in the environment before conducting the experiment and according to that, the odors are distributed in the room.

# **4.1.3 Participants and Sampling Method**

Criteria have been defined for participation in this experiment, which were: men and women between 25 and 50 years old, right-handed, non-smokers, with academic background (student, faculty, or academic staff) without any special disease (heart disease, mental illness, migraine, Epilepsy), without any problems or respiratory diseases (anosmia, hyposmia, nasal polyp, allergies, etc.), not having allergies to smells, perfumes, flowers, foods, without suffering from Corona disease and sufficient English language skills. Epidemiological studies have revealed that the olfactory function of individuals over 70 years old decreases significantly or they suffer from olfactory disorders. Also, the olfactory function of individuals is at its peak in the age range of 30–50 years (Doty et al. 1984; Kondo et al. 2020; Boyce and Shone 2006). Since the Corona disease can affect the person's sense of smell for a period of several weeks or months, all the participants were selected from persons who were not infected with the corona disease, and if they were infected, their sense of smell had returned completely. The number of participants in this experiment was 14 individuals (7 women and 7 men) who randomly entered the room with the fragrance of the (Jasmine flower) and the smell of rotten orange peel.

#### **4.1.4 Protocols of Attend on the Experiment Day**

Some protocols have been defined for the attendance of the participants on the day of the test. To ensure that the olfactory perception (sense of smell) of the participants is complete on the day of the test they should not be suffering from a cold or COVID-19. In detail, these protocols include: not using any perfume, body/oral deodorant, and clothes deodorizers on the testing day, not suffering from colds, the flu, allergies, nasal congestion, allergies, etc. two weeks before the test until the testing day, not eating any food (especially spicy foods) for at least one hour before the test, not drinking coffee for at least one hour before the test, not chewing gum on the testing day, not brushing one's teeth for at least one hour before the test, not wearing any metal jewellery (earrings, bangles, bracelets, necklaces, etc.) in the testing rooms and not having any communication and electronic devices (Mobile phone, smartwatch, etc.).

# **4.1.5 Data Collection and Data Analysis**

This test is a mixed methods design. On this basis, various tools and techniques were used to collect data so that both quantitative and qualitative data, as well as subjective and objective data, could be recorded and analyzed. In each of the test rooms, after the participant spends 3 min without any activity, she/he is asked to perform one touch Stockings of Cambridge (OTS) psychological task. The OTS is a 10 min task of executive function based on the *Tower* of *Hanoi* task and designed by CANTAB and assesses spatial planning and working memory in an individual (Backx et al. 2020; Blum Redden and Grant 2018). After that, participants completed a questionnaire that includes questions about their emotions and perceptions of the space.

In addition, brain activity (EEG) and peripheral signals of participants were recorded in the test rooms, borrowing from neuroscience. This was in addition to the questionnaire and the semi-structured, open-ended interviews conducted at the end of the experiment. To record brain signals in this test, a 32-channel Emotiv EPOC Flex saline system was used, which was placed on the heads of the participants in the form of a cap to record brain signals wirelessly. The EEG electrode location of this system is the standard 10– 20 electrode placement system. Also, Empatica E4 wristband device was used to record peripheral signals (see Fig. 3). This device is worn on the wrists of the participants like a wristwatch to measure the data related to blood volume pulse (BVP), heart rate (HR), electrodermal activity (EDA) and skin temperature (SKT) and send them via Bluetooth to a computer or a mobile. By integrating qualitative and quantitative, subjective and objective data, we can have a more accurate and standardized analysis and improve interpretation of the impact of environmental odors on human emotions.

**Fig. 3.** Biosensors used in experiments to record brain and peripheral signals. A, Emotiv EPOC Flex (32-channel) for recording EEG. B, Empatica E4 wristband for recording aperipheral signals (heart rate, electrodermal activity, and skin temperature)

For EEG data pre-processing and processing, EEGLab which is a toolbox plugin for MATLAB is used. Examining the EEG topographic maps of all participants shows that the power of different bands exposed to pleasant (Jasmine flower) and unpleasant smell (Rotten orange peel) was different and most of these changes are related to the frontal part of the brain (see Fig. 4). The power of the theta band in the frontal part of the brain is increased when exposure to the fragrance of jasmine flower. Also, smell of jasmine increases the power of the alpha band in the frontal part of the brain, while exposure to the smell of rotten orange peel decreases the power of the alpha band in different parts of

**Fig. 4.** EEG topography map of one of the participants in the experiment. It shows the changes in the strength of different bands (theta, alpha, beta and gamma) exposed to a pleasant smell (jasmine flower) and an unpleasant smell (rotten orange peel) in the first two minutes

the brain. Morover, the power of the gamma band exposed to the smell of rotten orange peel increased in the frontal part of the brain, while the power of this band exposed to the smell of jasmine in most areas of the brain either did not change significantly or decreased. Since the increase in alpha and theta band activity indicates an increase in relaxation (Motomura et al. 2001; Burnett et al. 2004; Sowndhararajan and Kim 2016), it can be concluded that the fragrance of jasmine flowers increased the relaxation of the participants in this experiment. Examining the results of the questionnaires also indicates that the participants felt more relaxed and pleasant in the room with the smell of jasmine flowers.

# **5 Conclusion**

Based on what has been reported in the literature, it seems that despite the importance and impact of smell on various dimensions of human existence, the role of this environmental variable in the process of architectural design and urban planning has not been seriously considered and very few studies have been conducted to investigate the effects of ambient smell on humans. Most of the studies conducted in this field lack a rigorous framework and methodology. Also, many studies conducted in this field still use traditional methods and tools to collect data, and up-to-date tools and technologies in the field of neuroscience are less used. In addition, it seems that the research conducted in this field have not been able to use the potential capacities of simulation techniques and methods to design and conduct research in this field. To bridge the existing gap this research investigated the effect of ambient smells on humans with a structured experimental design and empirical criteria (taken from different disciplines), and utilizing tools and technologies that are common in the field of neuroscience for such a study.

Designing and conducting systematic simulation experiments to study the effects of environmental smells not only makes it possible to evaluate and analyze the results obtained and reproducibility of these experiments, but also can create a basic systematic standard framework for future studies. By examining and analyzing the results, the methodology and structure of the experiment can be modified and developed for future research. In the experiment conducted by examining the electroencephalography (EEG) data, we found out that exposure to the fragrance of jasmine (pleasant smell) and the smell of rotten orange peel (unpleasant smell) can cause implicit changes and effects (power of different bands). If the participants' brain signals were not recorded and only traditional data collection methods such as questionnaires, observations and interviews were used, it would not be possible to investigate the implicit effects and physiological responses of smells on participants. Therefore, it seems that this mixed method approach can give a more comprehensive and accurate view of the implicit and explicit effects of the smell of the environment on different dimensions of human existence.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Parametric Sankey: Interactive Mapping of Complex Material Flows for Urban and Architectural Design**

Jeffrey Huang(B) , Frederick Chando Kim, and Mikhael Johanes

École Polytechnique Fédérale de Lausanne (EPFL), Station 14, 1015 Lausanne, Switzerland jeffrey.huang@epfl.ch

**Abstract.** The mapping of flows in a city is essential for understanding urban systems and enabling the transition of the city into a circular economy. However, while tools for the virtual representations of physical *volumes* and *spaces* in urban environments have proliferated, effective tools for modeling the underlying *flows* are still missing. This paper discusses the development of Parametric Sankey, a tool for the trans-scalar representation of flows, and demonstrates its instrumentation in a research design process. The proposed Parametric Sankey tool overlays different flow categories (material, energy, labor, customer, waste, etc.) into one dynamic trans-scalar system, enabling interactive processes between analysis and intervention, and providing thereby an experimental interface to envision future circular cities and architectures.

**Keywords:** Parametric Sankey · Material flows · Trans-scalar · Circular cities · Grasshopper

# **1 Architecture of Flows**

The circular economy promises to change how society produces goods, how they are procured and delivered, and how waste is collected and re-entered into the value chain. Flows are fundamental aspects of the circular economy that are both spatial and temporal, yet the interaction between flows and space, in which the seamless organization of flows that plays primary roles in defining the situation of the site, are often left unexamined (Holmes et al. 2021; Korhonen et al. 2018; Hesse 2010). In this research, we examine architecture's and the city's role in developing a circular economy through the lens of flows.

A proper understanding of flows is the key to the comprehensive use of resources toward circular processes, which demand the application of systems thinking (Chen 2009). Material flow analysis (MFA) is a central methodology for quantifying the aggregate resources used, reused, and lost (Graedel 2019). Visualization plays an essential role in comprehending the pattern of material flows, allowing detailed analysis and refinement of the intervention (Iacovidou et al. 2017). Sankey diagrams are the most common representation strategy to visualize the results, showing the interlinkages in systems while retaining the holistic display of their complexity (Schmidt 2008). The representation is numerical and diagrammatic, allowing simultaneous qualitative and quantitative observations.

There were attempts in architecture and urban design to represent and incorporate flows in the design processes concerning flows of people, information, and urban transport (Ballantyne and Smith 2012; Delalex 2006). One example is Louis Kahn's study of Philadelphia, where he visualizes the movement of cars to propose a new traffic pattern to untangle congestion (Kahn 1952). Several strategies have been developed to map pedestrian, private, and public transport flows on an urban scale (Dovey and Ristic 2017). The notion of flows is closely correlated with the performative aspects of architecture, where the formal and material configurations will dictate and be shaped by how the building's heat, energy, water, and wind flow (Cody 2017). Flows of people are also studied as the objective is to improve building safety and optimize the architectural experience (Derix 2014; Schaumann et al. 2019).

The interface between flows and architecture/urban context can be conceptualized as a 'space of flows,' where the 'purposeful, repetitive, programmable sequences of exchange and interaction between different physically disjointed positions' are happening (Castells 2010, p. 442). The flows, while dynamic, have inherent predictability and manifest as the city's infrastructures, networks, and architecture (Mangelsdorf 2013; Weinstock 2013). To understand the city's multi-scalar interactions with flows, we recontextualize the Sankey diagram, a scientific representation of flows, into a parametric modeling environment in an urban and architectural context. By finding the intersection between architecture, urbanism, and environmental science, this paper showcases the development of Parametric Sankey tools for addressing different flows in a design process and demonstrates its instrumentation in a research design studio. The Parametric Sankey tool aims to understand flows in the urban and architectural context and identify potential interventions contributing to the transition to a circular economy.

# **2 Parametric Sankey for Mapping Complex Flows**

Our research explores how flows can be represented interactively in different scales and situations, from pure diagrammatic processes to urban and architectural scales. We investigate the mapping and intervention of flows using a unified parametric Sankey tool in Grasshopper to facilitate trans-scalar flow analysis and development. Parametric modeling allows interactive exploration by establishing the association between sets of different parameters in which the changes of the parameters will reflect on the reconfiguration of the model outcome (Oxman 2017). This paper discusses the development of Parametric Sankey for addressing different flows in a design process and demonstrates its instrumentation in a research design studio. Figure 1 shows the proposed methods for the Parametric Sankey, which includes (a) collecting data to understand the processes of material transformation, (b) interactive flow modeling with different nodes of identified processes, (c) contextualization of flows in the neighborhood, and (d) architecturalization of flows in the building.

**Fig. 1.** Parametric Sankey, the proposed method for flow-driven urban and architectural design.

#### **2.1 Case Study: Mapping Sewoon Sangga Flows**

We use Sewoon Sangga as the case study to demonstrate the application of the interactive Sankey flow mapping tools in the design process. The objective was to examine how the Sewoon Sangga building complex, a 1 km long brutalist architectural relic of the 1960s in Seoul, could be revitalized and act as a catalyst for the transitioning of the surrounding district into an exemplary circular economy through the understanding of dynamic flows. This urban manufacturing and distributing neighborhood became the platform for understanding the current ecosystem and investigating the potential of Sewoon Sangga to participate in imagined circular manufacturing processes as a program for architectural interventions.

Over time, a bottom-up culture has emerged in this mixed-use commercial and residential complex, with artisans and entrepreneurs appropriating the megastructure. The uniqueness of Sewoon Sangga has been in creating an urban ecosystem with the Euljiro neighborhood that merges the manufacturing, material distribution, product, and repair culture. The diverse range of small scaled artisans with various skills and shop owners co-exist in cooperation along the organic alleyways as an urban structure to create specialized clusters such as printing, publishing, metalworking, tool manufacturing, jewelry making, electronics, precision machinery, and more (Baeumler 2016). The character of Sewoon Sangga and Euljiro is rapidly changing, and the injection of new capital is in the process to transform the city.

The mapping is started by investigating different materials flowing in and out of the market, from plastics, garments, plants, foods, electronics, and all of the derivatives. From these observations, preliminary material flows were created based on MFA principles by representing processes as collections of nodes where the flows of different materials pass through the process as the material turns into products and waste (Graedel 2019). The collected data identified the multiplicity of flows involving the flow of materials, waste, and energy (Fig. 2).

The identified processes provided a baseline for modeling the Parametric Sankey in which the interactivity between these processes could be explored and intervened. The parametric flows are juxtaposed with the existing system through a trans-scalar approach from processes, cities, and architecture towards a circular design.

#### **2.2 Parametric Sankey Tool**

The Parametric Sankey tool represents flows as a collection of nodes as processes and directed edges as material flows. We developed Grasshopper components to allow the

**Fig. 2.** Investigation examines the processes of waste's material transformation into new products.

parametric modeling of flows using input, output, and loop components (Fig. 3a). The input component receives information about the flow from the output component and draws the visualization flows in Rhino's model space. The flows are parameterized by flow properties such as name, volume, and color and geometrical properties such as node's location, direction, and working plane. A utility component facilitates feedback loops since Grasshopper does not support a recursive data stream. The developed components are designed to work in 2D and 3D modes using single or multiple working planes, allowing the multidimensional adaptation of parametric Sankey.

A process is represented by a node, defined by a cluster consisting of multiple input components and an output component (Fig. 3b). Multiple input flows are handled by cascading input components which accumulate the information to the output component. The output component receives parameters to define the distribution of flows to different channels. The flow is created and managed by connecting the node's output with the other node's input and defining the channel and distribution parameters (Fig. 3c). The looping flow information is fed back to the previous node using the loop component. The parametric model allows us to interactively explore different distributive and connectivity patterns through Rhino's modeling space (Fig. 3d). The modeling of the flows should follow the law of conservation of matter (Brunner and Rechberger 2004). The process node represents the transformation of materials from the input to the output. The processes are linked by flows that are measured in mass per time. The used units depend on the spatial and temporal scale of the described system, tons per year are commonly used in practice.

# **3 Results: Trans-Scalar Flows**

We model the flows in diagrammatic, urban and architectural scale in the design process. The modalities of Parametric Sankey for each representational mode are described below.

#### **3.1 Flows as Interactive Diagrams**

The Parametric Sankey overlays different materials and waste flow with other resource flows, such as energy, heat, and water (Fig. 4). The diagram is composed of processes

**Fig. 3.** Parametric Sankey component diagrams.

as nodes that transform the materials from one to another, as directed edges represent the flows. The color represents the layering, differentiation, or categorization of flows. The edges' size and color gradients reflect the flow's quantitative dimension. The node's position and the flow direction are reconfigurable, allowing the different formal configurations of flows to be explored, thus offering a way to disentangle complex flow into different typologies, such as merging, splitting, looping, and overlaying. Some nodes serve in a specific line of flow, whereas some nodes combine and intersect different flows altogether.

In architecture, one of the utilities of the diagram is to anticipate new organization possibilities (Allen 1998). Parametric Sankey allows architectural thinking to map complex material flows and explore the spatial relationship. The 2-dimensional constellation of distributed nodes gives an anti-hierarchical relationship in the material processes, decoupled from the physical manufacturing or logistical spaces. The resulting map visualizes the relative proportion of the flow. Any additions or modifications to the current flows will cause changes in the subsequent flows. It enables an intuitive understanding of the complex flows and interactive processes between analysis and intervention, providing experimental ground to envision a circular system. The parametric diagram recalls the idea of diagrams as proliferation instead of reductive machines (van Berkel and Bos 1998), departing from the conventional use of the Sankey diagrams as static representational tools.

**Fig. 4.** Parametric Sankey from the various material processes shows flows' layering, branching, merging, and looping. Work by Auguste Pachoud, Alexandros Trivizas, Mikaël Rey, Lorenzo Simontacchi, Anna Hausel, Isabelle Nguyen, Amélie Gaillet and Laura Guerreiro.

#### **3.2 Flows as Urban Assemblages**

Incorporating Parametric Sankey into an urban environment proposes an alternative view of urbanism as it favors the representation of behavior, situation, and interrelation over areas, locations, and territory (Batty and Cheshire 2011; Hesse 2010). The tool makes the analyzed processes tangible by depicting the city as a collection of flows where the places, buildings, and neighborhoods are the emergent of the productive processes (Dovey et al. 2017). The abstract processes are given spatial and material realities, giving a different reading to the overall flows. Sometimes, one process is distributed to different sites, and the logic of geographical proximity is now taking part. The resulting maps show the capacity of the Sewoon Sangga as a catalyst of a circular economy where waste and recycled materials are sourced and distributed around the neighborhood (Fig. 5). The flows show the interconnectivity between different places concerning a particular process and situate Sewoon Sangga as a strategic structure for future transformation.

**Fig. 5.** Parametric Sankey recontextualizes flows in the urban fabric. Work by Auguste Pachoud, Alexandros Trivizas, Flavio Nogueira Pereira, Bryan Marques Soares, Lucile Charamel, Léa Guillotin, Ilayda Makas and Ecenaz Ozkorkut.

#### **3.3 Flows as Architecture**

The urban flows are folded from the ground to the section plane of the building, engaging the flow entering and exiting the building, connecting the flow from the larger urban context to the architectural scale (Fig. 6). Representing flow in architecture brings forward the capacity of diagrams as 'contingent descriptions of possible formal configurations' (Allen 1998, p. 16), showcasing the dialogue between architecture and the other fields. The parametric Sankey graft the flows into the building's spatial realities, mapping the tensions and possible design interventions. The interactive framework allows the exploration of different configurations while maintaining the topological connectivity of the flows.

The interface between building typology and the flow's topology manifests into various architectural apparatus such as corridors, stairs, elevators, ramps, shafts, and chimneys, reconfiguring and revitalizing the obsolete typology. The flows challenge the existing condition of the Sewoon Sangga building. The transformative vertical of Sewoon Sangga into an automated-populated circular manufacturing and distribution results from careful spatial surgery of the building sections informed by the flows of waste, material, people, and other resources. These sectional interventions result from the architectural thickening of the flows to create formal acupuncture within the building.

**Fig. 6.** The architecturalization of parametric Sankey into building section. Work by Amélie Gaillet and Laura Guerreiro.

# **4 Discussion and Future Developments**

We developed a parametric modeling framework that allows the trans-scalar representation of flows. Flow diagrams can be intuitively developed and explored. Using case studies, we tested the capacity and limit of the tool and critically explored the potential future development to address circular cities. The interactive diagram allows the flows to be spatialized to make sense and reconfigure their pattern. The recontextualization of flows into urban fabric enables the reading of site interrelation, revealing the potential new nodes for circular processes. The architecturalization of flows confronts the obsolete typology with the need for new dynamics, which drives the reprogramming and revitalization of the building.

Parametric Sankey situates itself in the intersection of material flow analysis, urbanism, and architecture as an attempt to incorporate circular principles into design processes. By observing, analyzing, conceptualizing, and constructing with the flows through parametric Sankey diagrams, we better understand the city as a dynamic state in contrast to the static typological views. The flows are actively engaged in the architectural representational system as interactive diagrams in either abstract, urban, or architectural contexts. The tool lets the designer actively study, experiment, and discover various patterns of flow as sites of potential new relations and architectural inventions. The flows are represented as spatial and topological entities with which the architecture and urban design can engage. As a result, we get a unique perspective of investigating the urban manufacturing district in Seoul through the lens of material and energy flows from a linear to a circular economy. The formalization of these flows helps the conceptualization of the preliminary design stage.

Our framework is still in the early stage of development, which opens the diverging path of improvement to further explore its potential for architectural translation (Fig. 7). One possible future development is incorporating a parametric optimization process in the workflow by coupling the Parametric Sankey to particular objectives. For example, specific flows could be optimized to achieve different degrees of proximity with the other flows by strategically placing and orienting the nodes within the solution space. The current tool still uses explicitly defined parameters to model the flows, requiring manual iterative reconfigurations. An exciting direction to tackle this is incorporating graph-based modeling into the framework, where the flows can be formulated as an adaptive system reacting to certain conditions. Moreover, the current system can only represent flows on a specific static temporal scale. Further development is needed to represent the rich temporal aspects of flows.

**Fig. 7.** 3D Parametric Sankey in multiple working planes creates spatial 3d flows (**a**), and the volumeterized flows are printed with ColorJet Printing (**b**).

**Acknowledgement.** The results described in this research include contributions by the students of the "Circular Gangbuk Studio 2022" and the teaching team at the Media and Design Laboratory, by Marcela Velasco Delgado, Georg-Christoph Holz, Alexander Sadeghi, and Christina Doumpioti.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Apply Digital-Twin Model to Optimize the Planning of Equipment Pipeline System in the Laboratory Campus**

Gao Xinting and Zhuang Weimin(B)

School of Architecture, Tsinghua University, Beijing, China gxt22@mails.tsinghua.edu.cn

**Abstract.** Building Information Modeling plays an important role in laboratory design. The reasonable layout of the outdoor equipment pipeline is the key to supporting the efficient operation of the laboratory, increasing the flexibility of the laboratory space module, and planning a holistic smart campus space. However, the traditional BIM model lacks convenient visualization and interoperability in the early stage of the program and may lead to inconsistency. This paper aims to propose an integrated visual optimization model toolkit of the equipment and piping using the Rhino + Grasshopper platform. Based on this digital-twin model, the horizontal and vertical space required for the outdoor equipment piping system can be quickly calculated in the site planning stage. The workflow improves the efficiency and accuracy of equipment pipeline system design and reduces multiple design changes. After verifying the validity of the model through two virtual scenarios, it was demonstrated in a real laboratory campus. In the construction drawing stage, the toolkit was used to check whether the interspace of different professional pipeline meets the requirements. This paper expands the design concept, emphasizes the coupling relationship between pipelines and building space, and integrates the experimental and building space concepts throughout the design process.

**Keywords:** Smart park · Equipment pipeline system · Digital-twin model · Building information modeling

# **1 Introduction**

With the gradual development of laboratories, research buildings gradually begin to show strong characteristics of technological integration. The integration of building space and the integration of mechanical and equipment pipeline system of various disciplines, and the use of integrated design patterns to create constructive synergy of equipment pipeline, can lead to the continuous improvement of the intrinsic value of laboratory buildings (Bachman 2003). The unpredictable virus mutations, the worldwide vaccine development competition, and the growing demand for scientific research in biology and medicine have made the need for biosafety laboratory construction stronger than ever.

Nowadays, equipment pipeline system is a necessary part of the laboratory building. The reasonable way of equipment pipeline layout is the key to support the efficient operation of the laboratory, increase the flexibility of the laboratory space module and shape the image of the research building. The outdoor pipeline demand of biological laboratory is large, and the spatial relationship of pipeline system should be properly handled to avoid cross and other problems (Bachman 2003). The comprehensive equipment pipeline system is a structure and appurtenant facilities built underground in the city to collect various professional pipelines.

Existing equipment pipeline layout for laboratory buildings often involves a large number of interdisciplinary knowledge and complicated special content, resulting in a mismatch between drawings and process requirements. In addition, design changes are frequent. The existing research lacks the integration of equipment pipeline system as a design element into the preliminary spatial expression.

Using the developed toolkit, the integrated design concept was used to optimize the equipment pipeline system, aligning the different impact factors synergistically and enabling close collaboration between the various professionals throughout the project.

The contributions of this work are:


The rest of this paper is organized as follows. Section 2 provides a brief overview of the equipment pipeline system and focuses on rules for the arrangement of integrated systems. Section 3 introduces an integrated a visual model toolkit and explains how the digital twin workflow in order to improve computational efficiency and accuracy. Section 4 experimentally evaluates the optimization performance of the toolkit in both virtual and real cases. Section 5 concludes this study as well as some discussions.

### **2 Background**

#### **2.1 Laboratory Equipment Pipeline System Arrangement**

Outdoor pipe header system generally has two practices, railway engineering and equipment pipeline engineering. Common pipeline types mainly include water supply, gas, heating, rainwater, sewage, waste, cable, communication, etc. As can be seen from Table 1, although railway engineering and equipment pipeline engineering are both linear projects, they are constructed in different ways, with different economic costs, construction cycles, and expansion and maintenance methods. In addition, different construction methods have different constraints in the planning and landscape design stages.


**Table 1.** Characteristics of railway engineering and equipment pipeline engineering

### **2.2 Principle of Equipment Pipeline System Planning**

The demand for outdoor piping in biological laboratories is large. The spatial relationship of the system should be properly handled to avoid crossover problems. When laying out the equipment pipeline system should


The main purpose of space reservation for outdoor equipment pipeline system can contribute to maximize the use of underground space in the pre-design stage and increase the utilization rate of integrated pipe trench. This contributes to the reasonable landscape planning, in harmony with the underground equipment system (Fig. 1).

# **2.3 Related Work on Visualization of Equipment Pipeline System**

Current laboratory designs tend to ignore variability and interdisciplinarity. Most of the existing studies focus on late design and apply BIM models to simulate. The demand for biomedical research experiments is rapidly iterating, and the smooth conduct of experiments depends on the support of equipment. Laboratories for biomedical disciplines are a unique class of laboratory buildings that need to be easily changed, modified and expanded. Architects need to consider the functional needs of the building for future use (Table 2).

**Fig. 1.** Underground pipeline arrangement

**Table 2.** Summary of relevant equipment pipeline system planning proposed in the literature


# **3 Methodology**

Laboratory equipment pipeline system usually consist of more than 6 types of pipelines. Therefore, the calculation process of the system assembly and the reserved space is quite complicated. The toolkit is written using the GhPython platform to create a optimization model of the equipment pipeline system, which is intuitive and convenient.

#### **3.1 Overview of the Digital-Twin Workflow**

Define the logical framework of the digital-twin model. There are three types of input condition factors: (1) Pipeline variable parameters: building exterior contour line, pipeline type, and pipeline diameter. (2) Motorway variable parameters: motorway distance from the outer contour line of the building, motorway road width, and the minimum horizontal distance of the road edge line from the pipeline. Realization of the motorway automatic avoidance module. (3) Urban engineering pipeline planning requirements. And then apply GhPython Script to reproduce the distance between pipelines required in the specification, and realize the minimum horizontal and vertical clear distance filtered out according to the input pipeline type. Finally, the required horizontal and vertical spatial distances of the pipeline system are calculated by the merit-seeking algorithm module (Fig. 2).

**Fig. 2.** Diagram of the overall methodology of optimization design (Input: gray block)

#### **3.2 Equipment Integration Model Optimization Process**

First, select the horizontal and vertical clearances in the specification. In *Section A*, use GhPython Script to write the horizontal and vertical clearance cells separately to reproduce the specification. This enables the process of inputting the type of pipeline that can be automatically filtered to derive the horizontal vertical clearance of the pipeline. If there are multiple pipes of the same type, for example, if the result shows 1000 "TO" 1500, then it is proved that the List contains sequences. Decomposing it by Domain leads directly to a value containing two spacing values.

Then determine the variable values of the disturbance factor motorway in the following *Section B.* Based on the motor vehicle distance from the outer contour of the building, motor vehicle road width, road edge line from the minimum horizontal distance of the pipeline three values. Calculated to obtain the "road edge line from the outer contour of the building distance a" and "road edge line from the outer contour of the building distance value b".

In *Section C*, automatic motorway avoidance module. Ideally, the pipe header system should be laid avoiding the road. This will reduce the impact of the maintenance manhole cover on the traffic and will not affect the normal traffic during the maintenance. The values a and b from the construction are used to filter the resulting equipment lines in the XY direction. The magnitude of the "distance of the equipment pipeline from the outer contour of the building" and the "value a" are determined. If the value is less than a, it is considered that the category of pipeline meets the requirement in horizontal direction, otherwise the pipeline needs to be offset in XY direction (Fig. 3).

And then, calculate of the offset value in *Section D*. In order to reduce the horizontal space distance required for the equipment pipeline system as a whole, the first pipe to be offset coincides with the outer contour of the road by default. Therefore, the offset value of the first pipeline to be adjusted is the difference between the "value b" and the "distance of the first pipeline to be adjusted from the outer contour of the building". After determining the first pipeline to be adjusted by the offset, the remaining pipelines to be adjusted are OFFSET according to the previously determined minimum horizontal offset, and the final position of the pipeline system in the XY direction is obtained.

Finally in *Section E*, calculate the required vertical spatial distances for the system. The location of the pipeline in the XY direction is selected, and the initial location in the

**Fig. 3.** *Section C*: auto Lane avoidance program and analysis

XZ direction is obtained by sequentially moving the pipeline system according to the minimum vertical clear distance requirement value constructed by GhPython Script. In the vertical direction, we also need to consider that the sewage is discharged with a slope and the pipeline is buried deeper, so it is easy to affect the foundation of the building without a basement if it is too close to the building. Since there is a basement in this project, the depth of the sewage pipe does not have a significant impact. Since there are many solutions to meet the minimum vertical clear distance between pipes, Galapagos is introduced for the optimization analysis to calculate the minimum value of the vertical direction of the system. Based on the initial position in XZ direction, the solid model after the pipe is established. The height of the box is the vertical spatial distance (Fig. 4).

**Fig. 4.** *Section E*: calculation of vertical space required for integrated system

#### **3.3 Optimal Genetic Algorithm Evaluation**

There are several ways of arranging the system, and the toolkit is designed to select the optimal one. Take the relative position between 'water supply pipe-rainwater pipesewage pipe' as an example, after determining the position of the water supply pipe, there are 4 idealized layout patterns of rainwater pipe and 8 patterns for sewage pipe. After determining the burial depth of the first pipe line, the offset distance of the next pipeline can be derived based on the minimum vertical clear distance between pipes and the diameter. Positive and negative directions exist for each type of pipe line. The minimum spatial distance in the vertical direction of the pipeline system should be ensured with the burial depth. The optimal genetic algorithm is applied for the measurement (Fig. 5).

**Fig. 5.** Relative position between different kind of pipeline

In order to find the minimized distance objective function min(x, y), the mapping relationship with the fitness(x, y) function can be established through the following transformation, as follows:

$$fitness(\mathbf{x}, \mathbf{y}) = \begin{cases} Cmax - f(\mathbf{x}, \mathbf{y}), f(\mathbf{x}, \mathbf{y}) < Cmax \\ 0, f(\mathbf{x}, \mathbf{y}) \ge Cmax \end{cases}$$

Agentic algorithm battery within Grasshopper, the vertical spacing between pipelines is correlated with the vertical spatial distance of equipment pipeline system using Galapagos. The two ports Genome and Fitness correspond to the variable parameters in the function and the optimal solution, respectively. Through the selection, crossover and variation iterations of parameter configurations by the genetic algorithm, the gene parameter configurations with high utility function values in the results are retained. As the number of iterations continues to increase, the solution with the optimal value of the set objective function appears, and then stop the operation. The optimal configuration of the spacing parameters and the depth to be reserved underground can be obtained (Fig. 6). After the operation, the parameter condition that gives the optimized result is the value corresponding to the Genome port.

**Fig. 6.** Galapagos port output

# **4 Results**

The above steps are integrated by creating a visualization plug-in through the human UI interface and displaying the final optimization results directly in the main rhino interface (Fig. 7).

#### **4.1 Interaction Between Buildings and Equipment**

The iterative relationship between pipelines and buildings is divided into two aspects: positive and negative. First, the building has an impact on the pipelines. Through the automatically generated ranking system, different kinds of pipelines are arranged in order according to the preliminary plan in the planning stage. Secondly, the impact of pipelines on buildings. Considering the grouped buildings, attention should be paid to maintain the building spacing requirements when planning in order to keep the building group pipeline system from crossing. In addition, when arranging motor lanes, combined with the flow of the park, it is not appropriate to arrange multiple lanes when the building spacing is constrained. If it has to, the building spacing should be widened to avoid pipeline crossings (Fig. 8).

**Fig. 7.** Toolkit plug-in interface

#### **4.2 Interaction Between Landscape and Equipment**

Because of the overburden restriction, plants with root depth requirement cannot be arranged above the pipeline, which will cause damage to the service life. After determining the horizontal vertical distance to be reserved, the plant landscape and detailed site design can be reasonably arranged (Fig. 9).

#### **4.3 Validation Experiments and Practical Applications**

Applied to several different building blocks to verify the feasibility of the method in different situations. The results can be used as a reference when optimizing the solution. Firstly, two buildings in the virtual biological park were selected for the validation experiments. Building A, located in the northwest side of the park, is adjacent to the motorway on the north side. Building B, located in the middle group of the park, is adjacent to the motorway on the west side and has an old tree on the north side. The diameter of the type of pipeline is set and the resulting pipeline arrangement is checked. The resulting optimization was found to be feasible and could meet the planning and design requirements (Fig. 10).

**Fig. 8.** Interaction influence of building design and pipeline

**Fig. 9.** Adjustment of pipeline design based on landscape planning

The validation scheme is located in Nanhu, Jiaxing, China, a campus with a site area of 95,000 m2. As a bio-innovation laboratory, the complex pipeline systems were required to be considered throughout the entire process from planning to construction. Optimize the position of the manhole cover corresponding to the pipeline at the ground level, avoid the square in front of the entrance of the building and the car road of the park, and try to arrange it in the green landscape area.

The pipeline has two directions of movement, upward and downward, while meeting the vertical clearance requirements. The maximum vertical distance between all pipelines is set to 10,000 mm with an accuracy of 50 mm (Fig. 11).

**Fig. 11.** Create a concatenation of the two directions of movement

The Genome port connects the vertical distance thresholds between the control lines, and the Fitness port connects the vertical distance simulation results of the ductwork system. In order to obtain the minimum vertical spatial distance of the pipe heterogeneous system, the vertical net distance control end of Cluster is continuously changed. It gradually leveled off after the 19th iteration and stopped after the 131st iteration of the operation, yielding a required vertical distance of 500 mm for the equipment pipeline system. The result of pipeline distance and the iterative relationship of the visual toolkit are as follows (Table 3, Fig. 12).

# **5 Conclusion and Discussion**

This study developed the application of a digital-twin model to simulate smart park site plan and equipment pipeline system. The horizontal and vertical space required for the equipment pipeline system can be derived based on the input of building exterior contour lines, pipe types, pipe diameters, motorway widths, and engineering specifications. A platform for digital-twin model can be built to provide a reference for space reservation


**Table 3.** Vertical distance between pipelines

**Fig. 12.** Results of pipeline space minimization and optimization

of the pipe header system at the early stage of the scheme and facilitate the general drawing design.

Of course, there are still some limitations to this study. First, the application experience in real projects is insufficient. To address this issue, the feasibility of the tool was first verified by assuming several virtual environments, and then the actual application was performed in a project. There is a possibility of incomplete consideration due to the diversity in the construction of the project. Second, the optimal pipeline layout in this study is measured in terms of spatial economy rather than construction cost. However, the equipment pipelines in the actual project may be complex and extensive, and there are situations that are different from the visualization model. If we add a cost comparison module, we can balance the construction cost and spatial cost of different pipeline layout methods from multiple perspectives.

# **References**

Bachman, L.R.: Integrated Buildings: the Systems Basis of Architecture. Wiley, New York (2003)


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Modeling on Outdoor Thermal Comfort in Traditional Residential Neighborhoods in Beijing Based on GAN**

Pixin Gong, Xiaoran Huang(B) , Chenyu Huang, and Shiliang Wang

School of Architecture and Art, North China University of Technology, Beijing 100144, China xiaoran.huang@ncut.edu.cn

**Abstract.** With the support of new urban science and technology, the bottom-up and human-centered space quality research has become the key to delicacy urban governance, of which the Universal Thermal Climate Index (UTCI) have a severe influence. However, in the studies of actual UTCI, datasets are mostly obtained from on-site measurement data or simulation data, which is costly and ineffective. So, how to efficiently and rapidly conduct a large-scale and fine-grained outdoor environmental comfort evaluation based on the outdoor environment is the problem to be solved in this study. Compared to the conventional qualitative analysis methods, the rapidly developing algorithm-supported data acquisition and machine learning modelling are more efficient and accurate. Goodfellow proposed Generative Adversarial Nets (GANs) in 2014, which can successfully be applied to image generation with insufficient training data. In this paper, we propose an approach based on a generative adversarial network (GAN) to predict UTCI in traditional blocks. 36000 data samples were obtained from the simulations, to train a pix2pix model based on the TensorFlow framework. After more than 300 thousand iterations, the model gradually converges, where the loss of the function gradually decreases with the increase of the number of iterations. Overall, the model has been able to understand the overall semantic information behind the UTCI graphs to a high degree. Study in this paper deeply integrates the method of data augmentation based on GAN and machine learning modeling, which can be integrated into the workflow of detailed urban design and sustainable construction in the future.

**Keywords:** UTCI · Machine learning · GAN · Data augmentation

# **1 Introduction**

#### **1.1 Background**

Currently, most urban dwellers suffered from the urban heatwave events, including both systematic changes in climate such as warmer summers, and severity of extreme events such as heat waves [1]. A considerable number of studies have shown that the number of summer heat stress days suffered by Chinese urban residents has increased year by year in the past half century, which has caused significant urban health problems, especially for the life and health of the elderly. At the same time, research shows that the high temperature in the city even has a significant negative impact on the baby birth rate and the pregnancy safety of women [2]. It is estimated that there are about 24966 deaths related to heatwave in 2021, according to report of the Lancet [3]. Besides, heat-related labor loss, indirectly resulted in a loss of 1.68% of gross domestic product (GDP) in 2021 [3].

In the face of the growing urban heat problem, the number of government publications has risen each year in recent years. In 2021 alone, the number of papers related to climate health has increased by 3.7 times compared to the average annual number of papers issued in the past decades [3]. Academia has also paid close attention to the issue of urban climate comfort, especially the study of outdoor thermal comfort for hot summer climates. Over the past century, various models and metrics for thermal environment evaluation have been proposed for the study of urban environment as well as thermal comfort. Among them, the UTCI model, based on the human heat exchange mechanism and combined with the dressing model, integrates a variety of climate elements such as temperature, humidity and wind speed, and has the characteristics of multi-scale, multiarea and multi-climate generalization, thus becoming the mainstream evaluation index of outdoor thermal comfort today.

However, outdoor thermal comfort at the human scale has long been neglected in urban construction [1], especially in traditional neighborhood spaces, which is difficult to consider at the beginning of design. However, Urban neighborhood spaces are essential for residents by providing spaces for daily activities, of which the Universal Thermal Climate Index (UTCI) has an influence on space quality, where positive physical thermal comfort leads to more lingering and interactive activities, promoting healthy travel and improving quality of life [6].

Therefore, in this context, how to efficiently and rapidly conduct a large-scale and fine-grained outdoor environmental comfort evaluation based on the outdoor environment of urban traditional neighborhood spaces is the problem to be solved in this study.

### **1.2 Research Overview on UTCI in Outdoor Environment**

In general, research on urban-level outdoor UTCI is still in its initial stage in China, focusing mostly on macroscopic urban space, with relatively little research on microscopic human scale. The current research on thermal environmental comfort in urban space can be summarized from three aspects: research themes, research methods, and experimental means.


3. In terms of experimental methods, there are three main types of research: building a laboratory for human perception research [4], which allows for small samples of experimental research, more accurate and easier to control variables, but limited by the data samples and data sources; on-site measurement to collect climate environmental data, including temperature, humidity, wind speed, etc., and then calculate the UTCI values with the help of software, where this model usually limited to expensive cost such as time and money; the use of simulation software to simulate the virtual model of the site numerically, and then validated in the field, which is limited by the length of time consuming on software simulation.

#### **1.3 Research Gaps**

Although the current researches on outdoor thermal comfort are extensive, there are corresponding knowledge gaps, mainly as follows:


#### **1.4 Research Framework**

Compared to the conventional qualitative urban morphology analysis methods, the rapidly developing algorithm-supported data acquisition and machine learning modelling are more efficient and accurate, easing the problems of under-representation and interference by episodic factors in traditional research methods, and better model traditionally difficult non-linear phenomena [7]. However, machine learning models with superior generalization performance need sufficient data samples for training, in order to get more accurate prediction results.

For dealing with the above problems, we try to train a GAN model to replace numerical simulations, and related studies show that using GAN instead of numerical simulations for UTCI can improve the speedup by 120–240 times [8]. We propose a Grasshopper-based workflow (Fig. 1), combined with data simulation, augmentation and estimation.

**Fig. 1.** Workflow of this study

Specifically, we build a classical city model based on authoritative mapping data in Rhino/Grasshopper platform, and use Ladybug and Eddy 3D plug-ins to perform human-scale Micro-environment climate simulation; then Ladybug tools was used to calculate and generate UTCI images. Finally, based on the deep learning framework, we train a GAN model for future overall UTCI mapping of the city.

The remainder of this paper is organized as follows: Sect. 2 is a literature review of related researches, including the definition of UTCI and its application and researches about GAN. Section 3 illustrates the methods in this project about how to prepare the dataset required and how to train the GAN model. Section 4 describes the GAN prediction results and make discussions. Section 5 summarizes the main aspects of this article and proposes the possible application of the proposed model as well as the limitations and expectations.

# **2 Literature Review about Related Researches**

#### **2.1 The Definition of UTCI and Its Application**

The Universal Thermal Climate Index (UTCI), based on the multi-node dynamic thermophysiological UTCI-Fiala model [9], is used to predict human temperature and regulatory responses for combinations of the prevailing outdoor climate conditions. The UTCI is defined as the air temperature of the reference condition causing the same model response as actual conditions [10], which provides a human-based representation of the environment temperature, covering the whole climate range from heat to cold [11].

Compared with the physical temperature information, UTCI can more accurately distinguish the degree of human body's perception of cold and hot discomfort, which was widely used to be applied in tourism, urban planning, construction, etc., in different scales and climate zones [12, 13]. With the deepening of researches, some studies have been carried out in recent years on the regional applicability of UTCI [14, 15]. At the same time, researches on UTCI-related impact factors are also emerging. In general, these impact factors include climate factors, urban traffic factors, urban development intensity factors, micro-environmental landscape factors [18], etc.

#### **2.2 Researches About GAN**

Generative Adversarial Network (GAN), proposed by Goodfellow in 2014 [18], has rapidly created a research boom in the field of deep learning and image generation, and has been applied in various research areas. Based on this, various variants have been developed since then, such as DCGAN, WGAN, StyleGAN, etc. The GAN trains a generator network and a discriminator, where goal of the generator is to map a random vector to a realistic image, whereas the goal of the discriminator is to distinguish the generated and the real images [19].

Due to the advantages of allowing fast numerical generation by image transformation, GAN is applied in more studies, such as residential floor plan generation, building layout generation, garden layout generation, NDVI/NDRE prediction [20], precipitation nowcasting [7] and so on. Among them, the Digital Futures Workshop led by YAO et al. has explored GAN with generative urban design in numerous ways, and found that GAN has good applications in alternative environmental performance models [8].

#### **3 Methods**

#### **3.1 Model Generation Based on Rhino/Grasshopper Platform**

Rhino/Grasshopper, a parametric modeling platform, is the main modeling tool used by architects nowadays, which can effectively perform rapid model generation. GAN training requires a large amount of data, but the building of refined urban models is usually a complex process. At the same time, there was a problem of different scales in the collection of previous datasets, as the actual scales reflected by the input twodimensional images were uneven, resulting in inaccurate model predictions. In Huang's study [8], they proposed a fine method of "Prototype summary-Type derivation", to obtain a large number of city models analogous to the study area in a short period of time. However, the traditional numerical simulation of datasets involves simulating the environment of independent plots in a wind box, neglecting the correlation between the selected area and the surrounding environment.

Therefore, unlike Huang's study [8], we take into account the realistic characteristics and associative features of urban scenes. So, we choose 35 typical tracts (250 m \* 250 m) for modeling based on authoritative mapping data, and each tract satisfies the diverse characteristics of building layout forms. Our research area is the traditional historic district within the second ring road of Beijing, where we focus on this area for two reasons: on one hand, the study of the historic district, with its complex morphology, is relatively less studied on UTCI; and the outdoor thermal comfort of the historic district can influence the pedestrian spatial experience and promote the vitality of the historic district.

#### **3.2 Simulation and Calculation of UTCI Based on Ladybug Tools**

Ladybug software package, a collection of tools for environmental performance simulation on the Rhino/grasshopper platform, allows for the simulation of wind, light, heat and other climate parameters, in which outdoor comfort was evaluated using microclimatic and energy modelling with OpenFOAM and EnergyPlus, respectively. In this project, based on each simulation parameter, the final UTCI values were calculated using the Ladybug software to generate 35 slices of overall UTCI images. Referring to the principle of convolutional neural network, the UTCI images of the 35 whole city slices are then segmented using different sizes of convolutional kernels with different step sizes to ensure that the image dataset can satisfy the characteristics of multi-scale and front-back connectivity. Finally, we obtain 4500 paired picture datasets.

#### **3.3 GAN-Based Image Generation**

GANs were used to predict the outdoor environment comfort with full information, with learning global features instead of the detailed features of each object [8]. Based on the Tensorflow framework, we train a pix2pix adversarial network model for fast prediction of UTCI values, which can effectively reduce the time of environmental performance simulation. Pix2pix, one of the GAN models, conduct image-to-image translation with paired training data.

Finally, we perform data enhancement on the dataset, and images are panned and cut in four directions to achieve an 8-fold data enhancement, resulting in 36,000 data samples. The pre-trained model is then invoked to train the pix2pix generative adversarial network model, based on the TensorFlow framework. We divided the data set into training, test and validation set, in the ratio of 7:1.5:1.5, where the model was trained on the training set, and the robustness of the validation set and the model performance of the test set were evaluated.

# **4 Results and Discussion**

In this study, the information in the Fig. 2 shows that the training process of the pix2pix model gradually converges with the increase of the number of training iterations, and the mutual game process between the discriminator and the generator in the model training process can be seen from it. The generator loss increases slightly in the initial stage, and between 280 and 600 K iterations, the generator loss fluctuates up and down around 0.308, but increases after thousand iterations. The loss of the discriminator function gradually decreases with the increase of the number of iterations, and the model gradually converges after about 280 thousand iterations, with loss of the discriminator as around 0.3, but decreases after 400 thousand iterations. From the whole process, the model began to converge when the model iterated to 280 K, and after 600 K generation, the model appeared overfitting. From this, it can be seen that setting the iteration number to 300 thousand generations is more appropriate, so resetting the total number to 400 K generations for model training. The entire training process uses RTX3090 GPU, and the training process takes about 12.5 h.

**Fig. 2.** Training loss curves of generator and discriminator

Figure 3 shows the output results of the model on the test set, from which it can be seen that the predicted images can almost meet the performance and fineness requirements of the project, and the GAN model has good results in grasping the relationship and structural pulse of the building layout and UTCI as a whole. The model has excellent prediction performance for the layout of the enclosed building compound in the selected area, especially for the UTCI prediction of the north side of the building and the larger building courtyard. However, the prediction ability needs to be improved for the highly dense and overly complex building layout scenarios.

In order to further compare the effectiveness of the pix2pix model, we trained the cycleGAN model using the same dataset. The deployment method of the model dataset was the same as above, and a total of 20.4 h was spent with the using of RTX3090 GPU. The model eventually converged after 100 epochs, and model prediction results on the test set are shown in Fig. 4. Overall, compared to pix2pix model, there is a certain gap in details, which also proves that strict image-to-image transformation method of pix2pix has better performance.

Overall, the pix2pix model has been able to understand the overall semantic information behind the UTCI graphs to a high degree. Although this study is limited by time and computing power, and no more iterations are set, the model converges well so far, while reducing the over-convergence of the model caused by over-training.

280 P. Gong et al.

**Fig. 3.** Examples of model performance on test set

**Fig. 4.** Examples of cycleGAN model performance on test set

#### **5 Conclusions**

With the support of new urban science and technology, the bottom-up and humancentered street quality research has become the key to delicacy urban governance. In this paper, we propose an approach based on a generative adversarial network (GAN) to predict UTCI in traditional blocks. 36000 data samples were obtained from the simulations, to train a pix2pix based on the TensorFlow framework. After more than 300 thousand iterations, the model gradually converges, where the loss of the function gradually decreases with the increase of the number of iterations. We can clearly see that the pix2pix model has a high grasp of the relationship between the architectural form of historical ancient city blocks and UTCI. With the help of this model, we can quickly predict the fine-scale UTCI at the urban block scale in Beijing. Based on this, on the one hand, we can identify the overheated and uncomfortable areas in the ancient city in the historical ancient city and formulate more accurate policies. On the other hand, data mining can be used to explore the relationship with other urban factors. Overall, the model, learning from data to depict non-linear relationships between input parameters and output metrics, has been able to understand the overall semantic information behind the UTCI graphs to a high.

Compared to other studies on the use of GAN in built environments, the contribution of this article lies in: firstly, the processing of the dataset used for training first involves modeling historical ancient cities based on official data; secondly, each sample data is captured from fragments on a large-scale simulation result image, so each sample takes into account the influence of the surrounding environment; thirdly, the data interception method uses simple segmentation and sliding window interception to ensure the continuity of the dataset; to ensure the consistency of input data, each image proxy a 50 m \* 50 m block. Therefore, the model built in this study is limited to predicting at a 50 m scale to ensure persuasiveness.

Of course, the drawback of this study is also very obvious: the model training process lacks the necessary correction mechanism, and there is still a risk of overfitting the model without actually collecting data for testing. This is also the focus of our next research, including the control process of model training, the improvement of data types, and model correction based on actual data. Moreover, if the computing power allows, we can check whether there are fluctuations in the convergence of the model under more training iterations.

The key to future research lies in model evaluation. In addition to the Ineption score/FID (Fréchet Inception Distance), the next step is to construct a scale that can be easily understood by subjective experience.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Characterizing the Solution Space of Building Shading System Through Computational and Parametric Feed-Forward Design Approach**

Qi Zhang1,2, Linxue Li1(B) , Nan Ma3, Yunxiang Shan4, and William W. Braham2

<sup>1</sup> College of Architecture and Urban Planning, Tongji University, Shanghai 200092, China lilinxue@tongji.edu.cn

<sup>2</sup> Department of Architecture, University of Pennsylvania, Philadelphia, PA 19104, USA

<sup>3</sup> Department of Civil, Environmental, & Architectural Engineering, Worcester Polytechnic Institute, Worcester, MA 01609, USA

<sup>4</sup> Tongji Architectural Design (Group) Co., Ltd., Shanghai 200092, China

**Abstract.** Building shading systems play an important role in controlling solar heat gains, which can mitigate the impact of climate change on indoor environment. Effectively reducing cooling demand in summer and heating demand in winter requests further development of seasonally regulated shading systems. The main problem is to find a way that consumes less energy while reaching better comfort, which demonstrates potential for conducting an extensive search to parametrize configuration of shading. This paper aims to investigate the effects of building shading systems on energy and thermal performance in different seasonal conditions through a parametric design method, using Baoshan station in Shanghai, China as a case study to establish a baseline model and test different shading components' direction, depth, spacing, and tilt angle. The method explores a large solution space at the beginning of design, establishing a variety of approaches that can inform the architectural design team. The results showed that a proper passive shading system can reduce energy consumption by about 13% while thermal comfort meets ASHRAE 55 standards. This finding indicates the possibility of improving the indoor thermal comfort while lessening building energy consumption.

**Keywords:** Shading system · Building energy consumption · Parametric design · Solution space · Seasonal regulation

# **1 Introduction**

The IPCC report shows that the average global temperature increased by about 1.09 °C between 2011 and 2020. The frequency of heat wave events tends to in-crease along with climate change [1]. High temperatures can cause an uncomfortable indoor environment and threaten human health, while also causing increased use of air conditioning and increased energy consumption. To mitigate this impact of global warming on indoor thermal environment, building shading systems play an important role in controlling solar heat gains [2]. However, building heating and cooling demands vary from different seasons. More sun exposure is needed in winter compared to in summer. Effectively reducing cooling energy usage in summer and heating energy usage in winter requests further development and adoption of seasonally regulated shading systems.

This gap has been receiving increasing attention in the archived literature which tends to be solved by using smart control with automated shading system. Experimental studies have shown that shading systems integrated with lighting controls have a potential of more than 30% energy consumption savings in commercial office spaces [3]. Simulation studies have also shown that solar shades can reduce cooling energy savings by up to 50% [4]. The main problem, however, is to find a way that consumes less energy while reaching better indoor thermal conditions. The increasing complexity of building systems and growth of high performance computing capabilities has increased the use of computer simulation programs for the prediction of building energy consumption [5]. This demonstrates potential for conducting an extensive search to parametrize configuration of external shading system via digital modelling and simulating the energy consumption and daylighting, to identify the most appropriate solution family.

Based on this, this paper aims to investigate the effects of building shading systems on energy and thermal performance in different seasonal conditions through a parametric design method using Rhino, Grasshopper, and its plug-ins Ladybug and Honeybee. It focuses the design method on the features that architects control early in the design process. Preparing a large family of studies informs the architect about different successful approaches in the climate in a process sometimes called "feed forward" [6].

# **2 Method**

The study introduces a "feed-forward" design method for modeling and iterative optimization of building shading systems through parametrization. The method is based on the generative design tool Grasshopper, which has been widely used for parametric modeling in the architectural field. In particular, a large number of design scenarios for shading components models are parametrically investigated using the Ladybug and Honeybee environment plug-ins that are shipped with Grasshopper. Ladybug can import standard EnergyPlus weather files, providing interactive visual graphics that enhance the integrated decision-making process, while Honeybee gives the same design options as EnergyPlus, Radiance and Daysim, among other proven energy and daylighting engines that integrate with the Grasshopper interface to provide instant feedback on designs.

The workflow is that, firstly, a baseline building model is established based on the preliminary architectural design scheme, and all the parameters including geometry, orientation, thermal performance of the envelope are set, and simulations are run to obtain energy consumption results as the baseline reference. Then, based on the baseline model, the shading system model is established with design parameters including components' direction, depth, spacing and tilt angle, and the range of variables is set for parametric iterative study. Finally, sensitivity analysis is conducted based on the simulation data to find the relationship between design variables, energy consumption and thermal comfort (Fig. 1).

**Fig. 1.** The workflow of parametric design method for modeling and iterative optimization of building shading systems.

#### **2.1 Build the Model**

This study uses Baoshan station in Shanghai, China as a case study to establish a baseline model, which has a total of two floors with the footprint of 76.6 m2. The study simplified the building to the most basic box model, containing only the basic components such as walls, floors, roofs, and windows (Fig. 2).

The shading system in this study is external panel shading, modelled as shown in the Fig. 1, with adjustable rotation angle of the shade panels. In addition to the horizontal shading shown in the figure, vertical shading is also considered in this study. The set parameters of the model include depth, spacing and tilt angle, where depth represents the width of each panel, spacing represents the distance between two panels, and tilt angle represents the rotation angle of the panels. Take horizontal sunshade as an example, the panel is horizontal when the angle is 0°, rotates downward when the angle is greater than 0°, and rotates upward when it is less than 0°.

**Fig. 2.** Baseline building model and shading components model.

#### **2.2 Parameter Variables and Metrics**

The parametric iterative study process includes testing different shading panels' direction, depth, spacing, and tilt angle. Table 1 shows the setting of parameter variables and their range. The depth and spacing both range from 0.2 to 1.0 m, with tilt angle ranging from −75°–0° to 75°.


**Table 1.** Parameter variables and ranges

According to the building design scheme, there are three directions of windows, east, north, and west, and the study tests the effect of different directions of shading system on the building separately. There are 550 combinations of simulations in each direction, for a total of 1650. Through parameterizing a variety of design variables, we use the following evaluation metrics to assess the performance of shading system: energy consumption, thermal comfort.

# **3 Results**

#### **3.1 Parameter Iterative Studies**

#### **3.1.1 Baseline Results**

Firstly, the energy consumption of the baseline model without shading system was simulated, and the results are shown in Fig. 3. The annual energy consumption intensity (EUI) is 297.70 kWh/m2, of which the cooling EUI is 118.48 kWh/m2, mainly concentrated from June to September, and the heating EUI is 123.90 kWh/m2, mainly concentrated from December to March. Therefore, this study will mainly focus on the cooling energy use from June to August and the heating energy use from December to March.

**Fig. 3.** Monthly heating and cooling energy intensity of baseline model without shading system.

#### **3.1.2 Solution Space**

In order to integrate all results into one large solution space and analyse design variables, the open-source tool Design Explorer was used to exploring multidimensional parametric studies, thus can meet the design needs of different architects facing different design scenarios. Figure 4 shows the whole result sets for (a) east, (b) north and (c) west, for a total of 1650 cases. The first four items are the input variables, solar panel orientation, depth, spacing and angle, while the last 11 items are output results, including annual total EUI, annual heating and cooling EUI, monthly heating EUI from December to February, and monthly cooling EUI from June to August.

The result shows that shading on the east direction has the most significant impact due to the larger window area, with a minimum annual energy consumption intensity of 266.356 kWh/m2, a 10.53% reduction. The minimum value of annual energy consumption simulated for the north shading is 288.21 kWh/m2 and the minimum value for the west is 291.01 kWh/m2. The three groups of results showed a common feature that the shading system had a much greater effect on the cooling EUI than the heating EUI.

#### **3.2 Sensitivity Analysis**

#### **3.2.1 Effect of Depth and Spacing**

The results of the parametric studies allowed us to figure out the effect of each variable on energy consumption and thermal performance. Figure 5 provides correlation between shading panels depth, spacing with energy consumption. Seasonal regulation is not considered in this section and annual total EUI is selected as the evaluation index for analysis, due to the dimensions of shading panels are generally fixed and do not change with the season. In this parametric study, keep the tilt angle is 0°, and the depth and spacing variables are shown for 0.2, 0.4, 0.6, 0.8 and 1.0 m. A total of six scenario were

**Fig. 4.** Simulation results set: **a** East direction. **b** North direction. **c** West direction.

considered, including horizontal shading and vertical shading in the east, north and west directions.

The result shows that for the same depth, an increase in spacing leads to an increase in energy consumption. The impact is much greater on the east side than on the north and west sides, due to the large window openings on the east in the case chosen for this study. Horizontal shading and vertical shading also exhibit different characteristics. For horizontal shading, as the spacing becomes larger, the effect of depth also becomes larger. The opposite is true for vertical shading, as the spacing increases, the effect of depth changes decreases.

In order to study the correlation between depth, spacing and annual EUI more accurately, the study used the spacing to depth ratio (S/D) to evaluate, and investigated the correlation between S/D and total EUI. Figure 6 shows the results. When the S/D is the same, the energy consumption is basically the same despite having different depth and spacing. The results indicate a consistent trend that as S/D increases, the annual EUI also increases, and the increase gradually decreases. When the value of S/D is between 0 and 2, its impact on energy consumption is greater, and the curve gradually flattens out when it is *>*2. Therefore, architects should control the S/D of the shading system to be less than 2 or even *<*1, and reduce it as much as possible under the premise of meeting users' lighting requirements.

**Fig. 5.** Correlation between horizontal and vertical shading panels' depth, spacing and annual EUI in three orientations: **a** Horizontal shades on east side. **b** Vertical shades on east side. **c** Horizontal shades on north side. **d** Vertical shades on north side. **e** Horizontal shades on west side. **f** Vertical shades on west side.

#### **3.2.2 Effect of Tilt Angle**

In this section, the correlation between shading panels' tilt angle with energy consumption was investigated. Seasonal regulation is considered in this section, as the angle of the shading panels is adjustable. Annual heating EUI and cooling EUI, monthly heating EUI and cooling EUI, were all used as evaluation metrics. On the one hand, heating and cooling demands in summer and winter are different and the requirements of shading system are different, when more solar energy is desired in winter, while more heat blocking is needed in summer. On the other hand, as the seasons change, the sun height angle changes, which leads to a change in the angle of the effective shading panels. In this parametric study, keep both the depth and spacing are 0.6 m, and the tilt angle variables are shown for −75°, −60°, −45°, −30°, −15°, 0°, 15°, 30°, 45°, 60° and 70°. A total of six scenario were considered, including horizontal shading and vertical shading in the east, north and west directions.

**Fig. 6.** Correlation between horizontal and vertical shading panels' spacing to depth ratio and annual total EUI in three orientations.

Figure 7 indicates the correlation between tilt angle with annual heating and cooling EUI. The effect of tilt angle on cooling EUI in summer is much greater than that of heating EUI in winter, with cooling EUI varies in the range of 35 kWh/m<sup>2</sup> and the difference in heating EUI is only about 7 kWh/m2. EUI varies with angle, and heating and cooling EUI show opposite trends.

For horizontal shading, in the east side, the lowest cooling EUI and highest heating EUI are achieved at an angle of 75°, and the highest cooling EUI and lowest heating EUI are achieved at −15°. When the angle is 45° the energy consumption is the second lowest, and at the same time, the indoor light quality can be guaranteed. Therefore, when the angle of sunshade is adjustable, it is a better design choice to set the angle at −15° in winter and 45° in summer. The change of angle between north and west facing shading indicate a smaller change in energy consumption, However, we can still see the lowest cooling EUI when the angle is 30°.

For vertical shading, in the east side, the lowest cooling EUI is at an angle of 75° and the highest is at 0°. The heating EUI is the lowest when the angle is 45° and the highest when it is −75°. When the angle is 60°, the energy consumption is low and the quality of light in the room can be guaranteed at the same time. Therefore, when the angle of sunshade is adjustable, setting the angle at 45° in winter and 60° in summer is a better design choice. Similarly, north- and west-facing shading brings less impact.

When the angle of the sunshade is not adjustable, considering both the heating demand in winter and cooling demand in summer, by comparing all the results of horizontal shading and vertical shading, the vertical sunshade with the angle set at 45° has better performance in both the heating EUI and cooling EUI.

**Fig. 7.** Correlation between horizontal and vertical shading panels' tilt angle and annual heating and cooling EUI in three orientations: **a** Horizontal shades. **b** Vertical shades.

Figure 8 provides correlation between shading panels' tilt angle with monthly energy consumption, including monthly heating EUI from December to March, and cooling EUI from June to September. Shading system in north and west have less impact on monthly energy consumption and can be designed with priority from the perspective of daylighting health. Shading in east has less effect on heating EUI and more effect on cooling EUI. For horizontal shading in east side, −45° to 0° are the angles that should be avoided, and the monthly cooling EUI is lower in the 45°–60° range. Similarly, for vertical shading in east side, −30° to 30° should be avoided and −60° and 60° are better choices.

**Fig. 8.** Correlation between horizontal and vertical shading panels' tilt angle and monthly heating and cooling EUI in three orientations: **a** Horizontal shades on east side. **b** Vertical shades on east side. **c** Horizontal shades on north side. **d** Vertical shades on north side. **e** Horizontal shades on west side. **f** Vertical shades on west side.

# **4 Conclusion**

The results show that exterior shading systems play an important role in reducing building energy consumption and improving the indoor environment. Based on a "feed-forward" design idea, a large solution space is obtained by modeling and parametric iteratively optimization of the building shading. And the relationship between the shading effect with the depth, spacing and tilt angle of the shading panel in three different directions of east, north and west was obtained by sensitivity analysis. This has a great significance for guiding architects in selecting design solutions. In specific cases, the results show that proper passive shading systems can reduce energy consumption by about 13% while thermal comfort is met. The parameterization of shading system design indicates the potential that can guide future building design.

# **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **OTTO: A Portable Urban Sensing Station to Survey the Energetic Footprint of Urban Microclimates**

Mark Balzar, Zeynep Aksöz Balzar, and Galo Moncayo Asan(B)

University of Applied Arts Vienna, Vienna, Austria galo-patricio.moncayoasan@uni-ak.ac.at

**Abstract.** OTTO (Orthographic Trans-territorial Operator) is a sensory device that collects data on the urban environment through multi-spectral sensing to study urban energy flows. We hypothesize that urban nature and energy are a combination of biological and technological factors that interact in a symbiotic environmental cycle. By examining their relationships, we can gain insight into the impact of climate change. OTTO aims to create a high-res survey of the urban environment by using a new combination of multi-spectral bands. This research maps the relationship between urban form and energy exchange to gain a deeper understanding of urban microclimates through multiple types of energy data.

**Keywords:** Urban data analysis · Multispectral imaging · Sensing · Monitoring

# **1 Introduction**

OTTO (Orthographic Trans-territorial Operator) is a device that helps to study and understand the urban environment by painting a landscape that not only records the visual but by capturing many other felt characteristics and unseen qualities in the immediate surroundings. It uses various sensors to survey, map, observe, and listen to the city and analyze the energy flows within. The goal is to understand how the living and non-living parts of the city interact and exchange energy and how the atmosphere affects these interactions. This research will contribute to a better understanding of how cities can adapt to the impacts of climate change. The device uses multiple spectrums of data to create a high-resolution map of the relationship between the city's form and materials and its energy exchange, revealing details about the city's diversity in microclimates (Fig. 1).

The main questions become how atmospheric effects influence the living and nonliving participants of the urban realm and vice-versa. Can we provide some new information to find strategies for adapting to climate change?

The current paper presents the hardware and software setup of OTTO. It explains the strategies of correlating multiple sensor data (i.e., Temperature, humidity, solid particles in the air) with thermal imagery and LIDAR point cloud scans to understand the influences of urban morphology in this changing climate environment.

**Fig. 1.** Diagram of applied methods

# **2 State of the Art**

The current cloud-based multi-spectral data includes satellite imagery and geospatial information on a global scale. It detects long-term global phenomena but doesn't address the immediate impact of climate change on urban energy flows. The challenge of getting high-res data is tackled with portable sensors. Several initiatives aim to gather urban data using extensive sensor networks (Diez Ladera and Posada 2013). These devices can acquire data such as air quality, luminosity, humidity, and temperature. Though the urban environment is a complex whole that is in continuous flux through the interaction of their living and non-living participants throughout different timelines of the day that can be manifested in multiple layers of energy flows.

Current research, such as in Shade Lab at ASU, focuses on thermal exposure by acquiring weather data related to human behavior by correlating climatic data and pedestrian behavior (Kulkarni et al. 2022). In the case of OTTO, the influence of changing environmental factors such as air quality, surface temperatures, luminosity, and many others are correlated with the urban changes observed in 3d imagery of LIDAR scans, 360° photography, and through the audio analysis of the microphone recordings and sonar sensing.

# **3 OTTO Hardware Development**

#### **3.1 Sensing Devices**

The Core devices that regulated the organization of the design of OTTO through their technical and operating restrictions were:

• Ouster OS-1 64 (bit) LiDAR Sensor [angular vertical scan of 45° divided by the horizon = 22.5° up and down, 360° horizontal scan]

• Thermal Camera—FLIR C5—monodirectional 160 × 120 (19,200 pixels) true thermal imager, MSX® (Multi-Spectral Dynamic Imaging), 5-megapixel visual camera.

OTTO has been developed as a proof-of-concept prototype and includes budgetfriendly sensors and microcontrollers based on the Arduino platform.

The Sensors applied at the final stage of the design of OTTO are as follows:


The Arduino-based development of the project put many limits to the core dataset gathered through these sensory devices. For the purpose of refining the results, many different sensors have been tested. Integrating multiple hardware devices in the R&D process poses a significant challenge, as each device has unique physical specifications and coding requirements. Ensuring a stable data flow requires a meticulous approach to combining these elements in a way that accommodates the varying needs of each device. This can present a significant barrier to the success of R&D processes, as even minor inaccuracies in the integration of hardware devices can result in data disruption or loss.

To mitigate this challenge, it is crucial to thoroughly research and understand the requirements of each individual hardware device before attempting to integrate them. This includes examining the physical specifications, such as size, power requirements, connectivity options, and coding requirements, including any necessary software, and programming languages. By carefully considering these factors and conducting thorough testing throughout the integration process, it is possible to establish a stable data stream that accurately reflects the results of the R&D process.

Some sensors would give us only qualitative results combined with those that give us quantitative results. The assembly of gas sensors is representative of this case. Even though Digital light Sensors give us a range of lux sensing, they have to be individually set up for the different pre-set scenarios of direct daylight, ambient light, nighttime, and interior scanning.

#### **3.2 OTTO Design (Prototype)**

The following were the guidelines for designing the hardware skeleton and structure:


To meet the stated objectives, 3D printing with filament extrusion was determined to be the most efficient materialization, development, and production method. Various filaments, including PETG, PLA, and ABS, were evaluated and tested based on deformation, material stability, and porosity under heat and rain exposure. This evaluation process considered various printing scenarios and configurations, leading to the selection of an extruder utilizing PETG filaments. This method was deemed the most suitable due to its balance of budget-friendliness, speed, precision, and accessibility in different cities worldwide for replacement purposes (Fig. 2).

**Fig. 2.** Hardware iterations of OTTO. The first iteration where the modular body of the device was developed, contained the LIDAR sensor and environmental sensors. (Left) In the second iteration The modular body was extended to include the extended version of environmental sensors and thermal camera. The final iteration included rain protection and 3D camera (right)

The geometry of OTTO is based on a truncated octahedron segmented into its individual joints. The system is regulated by a single regular joint knot, which allows for technical extension in all directions and connection through screw nuts on the central axis. Originally it was tested to route the wiring through the axis of the knots themselves, but relatively soon had to balance the relationship of the size of the knots with the number of wires we are able to route through them. The more flexible and efficient way for prototyping turned out to be external routing. It kept OTTO's knots and general size relatively small, making dismantling for packaging easy. All Arduino Sensors and the Thermal Camera (on the central axis to the Lidar) have been organized around the same principle. Weatherproof closures were achieved by the careful routing of the wires and coverage of the electric contacts. The sensors themselves have individual custom caps to achieve rainwater drainage. The volume inside the truncated octahedron gave us space to position the Arduino microcontrollers as a central "heart" of OTTO which kept wiring at the minimum. Any other organization of the core OTTO microcontrollers incl. The Weathersfield outside the center increased wiring and put the "heart" into a vulnerable position while moving OTTO through the city. The Ouster LiDAR sensor was positioned on top of all other sensors to avoid obstacles during measurement.

# **4 OTTO Software Development**

#### **4.1 Serial Monitoring and Computer Communication Interface**

OTTO acquires data using three communication protocols that save the streams in three file types. Two Arduino boards with environmental sensors send the data via USB to a computer using a listener interface. The data is collected by an Arduino Mega and sent as a.csv file. A node-red-based protocol is used to process the data in the.csv file and write each spectral dimension as a column into an SQL database. Each acquisition is tagged with the acquisition location as longitude latitude, and acquisition time to enable the filtering of the data in the SQL database (Fig. 3).

**Fig. 3.** Visualization of the LIDAR point cloud and the overlay of nighttime temperatures

The communication protocol of the Lidar system was evaluated through two distinct methods. The LIDAR device comes with its own recording and visualization platform, which provides real-time access to the Lidar point cloud and calibration information of the sensor, including the IMU sensor. The platform timestamps each frame and stores a series of frames over a specified duration in a native format called "pcap". However, longterm recordings in this format can consume large amounts of data stored. Alternatively, a Python code based on the Ouster SDK was developed to directly access the LIDAR scanner via localhost. This approach enables the saving of necessary information, such as the XYZ coordinates of each point in the point cloud, in a.csv file, which can then be imported into any visualization interface for observation, visualization, and analysis.

The existing Ouster Studio Interface is a powerful visualization system. The recorded point cloud is easily executable to.csv. An additional interface provided by Open3d enables the users to visualize the point cloud in a custom web interface. The Open3d python package also comes with a machine learning interface that enables users to apply 3d object recognition tasks on the point cloud (Argueta 2022).

The thermal images were essential to visually compare the places of data acquisition from an IR spectral band. These were used to compare the urban heat properties of different locations within the same city. In this case, the thermal image is an important indicator of the impacts of the materiality on the overall heat behavior of distinct places.

The thermal image acquisition is conducted via an internal code from the computer, pinging the camera to save an image frame each second. The reason for longer acquisition periods is to detect the changes in surface temperatures throughout the time, record impacts of moving objects such as cars or humans, and map their influence on the further recorded data layers.

#### **4.2 Data Post-processing and Analysis**

#### **4.2.1 Post-processing Sensor Data**

The sensor data is analyzed by applying three different kinds of correlation approaches. The first approach correlates data dimensions in one location to understand the underlying relationship between the acquired environmental values. For such correlation, sensitivity analysis using regression was applied to analyze the sensitivity of each multispectral dimension to the other ones. In this analysis, the sensitivity of the gas values was calculated in regard to the changes in weather conditions such as temperature, humidity, wind speed, and direction.

The secondary correlation was conducted between different places to localize the environmental tendencies within the urban conglomerate.

The final comparison was the changes in the environmental conditions of the same area within the different time gaps of the day to understand the transmission of energy throughout the day and night.

#### **4.2.2 Post-processing Thermal Images**

Usually, a thermal camera acquires a data type called radiometric JPEG. This format inherits for each pixel 4 dimensions of information. Additional to R, G, and B dimensions, it also has an additional dimension for temperature. When directly acquired with the camera, one can access these channels through the FLIR's own software. However, when pinged via a laptop connection, FLIRs own system blocks the host from directly accessing the infrared sensor. To map the thermal information easily in post-processing all the images were acquired in grayscale with a preset upper and lower threshold for visualized temperatures. This way we can map the grayscale back to a numeric domain.

# **5 Results**

#### **5.1 Phoenix Case Study**

Phoenix is one of the hottest cities on earth. With an arid climate in the desert, the city is facing big challenges driven by heat. The extreme heat rising to 48 °C (Hondula 2020) during hot summer months does make outdoor urban life challenging. Within the greater urban area of Phoenix temperature differences of up to 12 °C (Hondula 2020) can be observed between different urban areas. This difference is not dependent on the building density, as a large part of the city is populated by single-family houses but rather on the materiality of built surfaces and the distribution of greenery and vegetation.

**Fig. 4.** Daytime (left) and nighttime (right) UHI maps and markings of the survey locations and their properties in regard to heat, vegetation and water surfaces

Selection of the survey areas was conducted by mapping these extreme zones within various satellite data sets provided by Google Earth Engine using QGIS. And by collaborating with local researchers to understand the demographics and existing strategies for mitigation of urban heat island effects (Fig. 4).

Eight areas for the survey were selected, each displaying different environmental and demographic properties. In the following sections, a summary of the results from this survey will be demonstrated.

**Fig. 5.** Comparison of VOC levels (left) in the hot zone of Glendale Phoenix with the implementation of the Cool Pavement project (represented in red), and a cooler area of Arrowhead Park Lake (represented in blue) with a large water body. Comparison of temperature difference in the same areas (right). These two diagrams illustrate how the use of cool pavement could potentially decrease environmental temperatures. However, it's important to note that through the radiation process, gases such as VOC are released into the atmosphere, which can have a negative impact on air quality

#### **5.2 Comparison of Extremes within the City of Phoenix**

The survey areas were divided into two groups of hot and cool zones. The comparisons were conducted by juxtaposing the exact dimensions of data in different zones. One important observation was that the hotter residential areas also lacked the greenery and shadow areas created by the vegetation. The socio-economic differences between the residential areas can be quickly observed in the vegetation distribution. The lack of green areas and shadow is resulting in the surface overheating of the surfaces and increasing the overall temperature of the area. The City of Phoenix undertakes actions to mitigate the heat and nighttime temperatures through thermal radiation. One of these actions is painting the asphalt white to decrease the heat gain and avoid nighttime heat radiation. One of the test areas was also selected as a survey location compared to the cooler zones' areas. In this comparison, an important observation was that the outdoor temperatures were lower compared to the cooler zones. However, when comparing the air quality, a large difference was observed in the amount of VOC (Volatile Organic Compounds). Which was dramatically higher in the hot zone with the painted pavement.<sup>1</sup> Based on these observations, it is assumed that the pavement's paint can release organic chemicals when exposed to heat. Even though it is decreasing the environmental temperature, it is also increasing the air pollution in the area. However, a larger data set of observations over a longer period in different seasons must be acquired and analyzed to make a precise conclusion (Fig. 5).

#### **5.3 Results of Sensitivity Analysis**

Through sensitivity analysis, the correlation between different data dimensions within a survey area were examined. In Phoenix, we observed a strong co-dependency between air quality and temperature, as well as between air quality and time of day. In hot areas with increasing temperatures, the release of volatile organic compounds (VOCs) also increased. Although some plants can release VOCs with increasing temperatures, a dense plant distribution in the hot areas of the survey was not observed. Therefore, we assume that the surface or coating materials could be related to the increasing release of VOCs.

Interestingly, a similar correlation was also observed in cooler zones with dense greenery. However, the direct correlation between temperature and VOC was not evident in cooler areas with less dense greenery. The complexity of the relationships between data dimensions makes it challenging to draw clear conclusions. Nevertheless, our research aims to explore further the correlations between VOC and temperature to gain a better understanding of this complex relationship (Fig. 6).

#### **5.4 Comparison of Day-Time Night-Time Surveys**

The survey locations were examined during both daytime and nighttime. These locations were surveyed and recorded during the early morning hours, before sunrise until half an hour after sunrise, at noon, and after sunset. In Phoenix, public spaces are typically used before the high temperatures hit the city, which means before 8 AM in the morning and after 7 PM in the evening. During the time in between, public spaces are rarely used. The data analysis showed that heat radiation at night is very high in residential areas due to the wide asphalt streets. After sunset, the surface temperature of asphalt remains at 40 °C (Fig. 7).

<sup>1</sup> VOC is compounds that are human-made chemicals that are integrated into paints, pharmaceuticals, or refrigerants (EPA 2022).

**Fig. 6.** This diagram shows the correlation of all the sensor dimensions, where red represents the highest measurements and white represents the lowest. It illustrates the correlation between different sensor measurements during a one-hour sensing event, providing a timeline of the data collected

**Fig. 7.** Temperature landscape of the same survey zone in night time (left) after sunrise (middle) before sunrise (right)

During the nighttime recordings, a constant air temperature of 37–40 °C was observed over an hour of the recording period. In the morning hours before sunrise, temperatures started at 28 °C and rose to 37 °C after sunrise.

The high temperatures also had an impact on wind speed, as gusts were observed to increase with rising temperatures. This may occur due to the movement of hot air that has been heated through radiation.

#### **6 Conclusions and Future Work**

But while the climate crisis was engineered in the past, it was mostly in the recent past; and the degree to which it transforms the world of our grandchildren is being decided not in nineteenth-century Manchester but today and in the decades ahead (Wallace-Wells 2019).

Climate change is a persistent issue that requires a comprehensive understanding to make informed conclusions. It is important to comprehend the relationship between the built environment, materiality, and human influence using architectural tools to adapt to its impacts.

One of the limitations of this research is the survey and observation time. The survey of Phoenix (as well as other survey zones which are not described in this paper) were conducted in the yearly extreme seasons. Though the seasonal extremes can also have their meteorological diversities. That means the hot season does not include only hot days and there can be meteorologic deviations. Therefore, surveying a large urban area within a short period of time does not provide a complete overview of the energetic events of the urban zones. Also, comparisons between locations within the same urban areas must always be conducted under similar meteorological circumstances. With the limited survey time finding a similar meteorological circumstance can be challenging. To overcome these issues, we suggest combining the acquired data with online available satellite and environmental information in the next steps.

Still, data post-processing steps are missing that will correlate the urban LIDAR data with the sensor recordings. One step in development is the analysis of the 3d point cloud to detect the changes within the survey area with the help of machine learning. This will help the team to detect the interchanges introduced through moving urban participants.

Another focus will be surveying other cities in extreme climate zones to learn about their challenges and mitigation strategies. The next survey will be conducted in Jakarta Indonesia.

# **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **A Parametric Approach Towards Carbon Net Zero in Agricultural Planning**

Wang Yueyang and Philip F. Yuan(B)

College of Architecture and Urban Planning, Tongji University, Shanghai, China philipyuan007@tongji.edu.cn

**Abstract.** This paper presents a new tool called the Space Data Generator, which is a parametric tool for organizing open spaces in rural areas. It can optimize the layout of buildings, solar panels, and agricultural planting spaces. While architects have been exploring ways to achieve net-zero carbon emissions in building design, it is equally important to attain a feasible carbon-neutral goal in rural areas. This is particularly crucial as 40% of the world's population resides in rural areas, and transitioning towards a more sustainable and efficient economy can bring about not only moral but also economic benefits through proper management [1].

The Space Date Generator offers a powerful spatial planning approach for optimizing and planning agricultural resources on any given land. This innovative tool utilizes a combination of remote sensing to generate precise maps of the land, providing a comprehensive understanding of its terrain and potential agricultural resources. With this information, farmers and land managers can make informed decisions about crop selection, irrigation, and fertilizer application, among other factors. By using the Space Date Generator, they can optimize the use of available resources and maximize crop yields, ultimately increasing profitability and sustainability in agriculture [2].

Overall, the Space Date Generator is a valuable tool for any farmer or land manager looking to make the most of their land and resources. Its ability to provide detailed and accurate data on the land's potential agricultural resources can help to streamline decision-making processes and ultimately lead to more efficient and sustainable land use practices.


The land area at hand will be subdivided based on its sun ratio, which is a relatively straightforward process. However, we are also interested in determining the value of excess vegetation that may grow in the allocated space. In this regard, the Space Data Generator can prove to be a valuable tool, not only for this particular scenario but also in other types of agricultural settings such as those involving a mix of livestock and crops. Additionally, it may be possible to use this tool to calculate the optimal harvesting of various plant species at different points in the seasonal cycle.

The Space Date Generator has the potential to offer valuable references for optimizing agricultural schemes. However, it must provide users with completely accurate results. Unfortunately, it currently cannot measure crucial factors such as soil type and moisture level, which are essential for agricultural planning. Despite this limitation, the Space Data Generator is a flexible tool that can be modified as research advances, allowing for more inputs to be added to improve its accuracy. Moreover, the Space Data Generator can provide guidance in various other areas based on the specific needs of the user. For instance, it can offer guidelines for traffic and urban design, among other demands. By leveraging this technology, users can access more precise and relevant information, enhancing their decisionmaking capabilities. As such, the Space Data Generator represents a valuable tool for various industries and sectors.

#### **Guidelines**:

**Keywords:** Carbon Net Zero · Bio-design · Urbanism · Genetic algorithm · Hybrid architecture

# **1 Background Introduction**

The Space Date Generator is robust tool that can provide a comprehensive and wellthought-out analysis based on various aspects of interest. The tool takes into consideration the user's preferences in a sequential order, with a primary focus on achieving self-sufficient carbon neutrality [3].

To accomplish this goal, the tool generates a plan that fulfills each household's nutrition and electricity needs. The electricity requirement is met by the solar panel installation, while the concept of pixel farming (Fig. 1) is utilized to generate the necessary nutrition. Additionally, the tool considers adding other factors, such as planting crops or raising livestock, to fulfill economic interests. The tool also takes into account the location of existing crops and solar panels to generate the pathway closest to the edge of the field, facilitating optimized movement paths. The land usage can be edited to meet specific needs, and the tool can keep specific areas empty based on the user's preferences. However, the generated result will not defy certain conditions, such as rivers, large stones, or trees, which must remain in their original positions. The tool will adjust the result based on the existing conditions, ensuring the optimal use of the available land.

The Space Data Generator is an incredibly powerful tool that is revolutionizing the way land usage is planned and resource arrangements are optimized. Leveraging its advanced capabilities to generate optimized layouts tailored to specific parcels of land, this tool offers invaluable guidance to a diverse range of users. Whether it's individuals seeking to maximize their own land utilization or businesses looking to generate additional income streams, the Space Data Generator empowers users with insights and solutions. By using the Space Data Generator to make the most of available resources, users can not only fulfill their own needs but also contribute to the achievement of carbon neutrality. This tool offers a practical means of achieving both personal and societal goals, providing a win-win solution for anyone seeking to make the most of their land. With its potential to transform land use practices and promote sustainable development, the Space Data Generator represents a major step forward for those seeking to balance economic, environmental, and social considerations in their decision-making [4].

**Fig. 1.** The image above shows how pixel farming works in real life, image credit belongs to (Pixel farming RoboticsLaagt 164286 LV Almkerk)

# **2 Preconditions**

The fundamental requirements for space data in the context of this program encompass a range of elements, including the site boundary and site form. These elements encompass various features such as rivers, roads, trees, and existing architecture locations, which are essential in determining the optimal placement of solar panels. Additionally, users have the flexibility to self-select other preconditions, such as house size and the number of households. Based on the energy needs of the household and taking into consideration the fact that solar energy has a higher energy conversion efficiency compared to the number of calories produced by plants for human consumption, the program automatically places solar panels in locations that are most energy-efficient.

After the solar panel is positioned, users can enter the parameters of the different types of plants they wish to grow, including energy conversion rates and market prices. The Space Data Generator then provides an overall calorie analysis based on the site conditions mentioned earlier. The analysis subtracts all the calories required by the user based on their input. If there is any surplus, the program determines how the remaining food can be sold in the market. In summary, the Space Data Generator utilizes various site-specific data to create a personalized farming plan that optimizes energy efficiency and food production. By combining user inputs with advanced algorithms, the program can help users achieve sustainable, cost-effective farming practices while simultaneously generating surplus crops for potential market sales.

# **3 Generating Housing Blocks**

The Space Data Generator is a novel approach that utilizes the household number as input to determine the optimal house type for a specific location. This innovative system incorporates various room types that are specifically designed to accommodate a particular number of families. Additionally, the size of the household options has a direct impact on power consumption and calorie requirements, which are determined using algorithm. The house types in the Space Data Generator are carefully designed to cater to different family sizes. These house types are optimized to provide adequate space for families of varying sizes, ensuring that each family member has the appropriate amount of living space. The system takes into account the number of families in a household and determines the ideal house type that can comfortably accommodate them.

In addition to determining the appropriate house type, the Space Data Generator also considers the impact of household size on power consumption and calorie requirements. The system utilizes the algorithm that takes into account the size of the household to estimate the power consumption and calorie needs. Larger households with more families may require higher power consumption and increased calorie requirements to meet the needs of all residents. Space Data Generator is a system that combines multiple factors, including household size, house type, power consumption, and calorie requirements, to generate optimal solutions for housing in a specific location.

#### **4 Set Building Coordinates**

Once the housing block type has been generated, the Space Data generator will proceed to conduct an analysis of the optimal location to place the house on the site. To begin with, the existing site will be divided into smaller pixel blocks, and each block will be thoroughly evaluated to determine its potential for achieving optimal sunlight exposure. Safety considerations will be taken into account, and any blocks that are located adjacent to a river or the site boundary will be excluded from further consideration. Furthermore, the decision of whether to orient the house based on the direction of the sun will be left to the discretion of the user. This means that the user will have the flexibility to choose the orientation of the house that best suits their preferences and requirements. By providing this level of customization, the Space Data generator aims to ensure that the generated housing design aligns with the specific needs and preferences of the user, while also taking into account safety and environmental considerations.

Once the housing block is situated on the site, the Program will commence the process of calculating the walking paths for each household. These paths enable residents to freely navigate the site and connect to any part of it. The path determined is the shortest distance from the building to the site edge. In the case of a triangular field, there would be three paths leading to the edge of the area. Each path is perpendicular to the road's edge, which is wide enough to accommodate a car's passage. The Space Data Generator will initiate the calculation of how to maneuver around obstacles such as trees or stones.

### **5 Analysis of Electricity Consumption of Residential Blocks**

As stated earlier, the central aim of this project is to offer a technological solution for achieving carbon neutrality. An effective architectural design can provide ethical solutions to address the changing composition of the Earth's atmosphere [5]. Additionally, we understand that the foundation of this ethical objective is to attain an excellent economic solution that yields a carbon-neutral outcome. From this perspective, if our tool can generate long-term economic benefits, it will substantially enhance the feasibility of its application across a wide range of contexts. To begin with, the Space Data Generator must allocate solar panels based on the annual electricity usage of households through calculations. This involves determining the electricity required for the "input" section of the program. Users can input their country, and the panel will display the average electricity output for that location. The program will then multiply this value by the number of households to estimate the total energy requirement by Grasshopper's Genetic Algorithm plugin [6]. The Space Data Generator will then convert this energy requirement into the number of solar panels needed, taking into account the specific efficiency of the panels being used. The default production efficiency is 250 KW/H per 3 km2, but this value can be adjusted to meet the specific needs of the solar panels.

With this information, the program can then determine the optimal placement of the solar panels based on the available sunlight. Initially, the panels will be positioned in a three-star configuration. If this location does not meet the energy demand, the panels will be shifted to a two-star configuration. The number of stars corresponds to the amount of sunlight available in the given location. It's important to understand that the Space Data Generator takes into account site classification based on different levels of sun intensity. Specifically, in areas with two stars, the light intensity is 75% of that in three-star areas. For instance, if 12 solar panels are installed in a three-star location, they can generate 3000 KW/H of energy. However, the same number of solar panels in a two-star location will only generate 2000 KW/H. Therefore, based on the Space Data Generator's calculations, a minimum of 16 solar panels would be required to generate sufficient electricity in this scenario.

# **6 Place Solar Panels**

After calculating the optimal location for solar panels in each region, the algorithm proceeds to determine the maximum distance from the solar panel to the edge of the site. We acknowledge that humans have a natural inclination to establish a closer proximity to plants than to solar panels, and this has been supported by numerous research studies [10]. Once the location of the solar panels has been determined, the Space Data Generator will then calculate the most suitable land for crop placement.

# **7 Rating Land Type**

To engage with this algorithm, the user must input their preferred type of plant. The algorithm will then automatically calculate the household's calorie consumption based on their population, and determine the total calories provided by each type of plant [7]. This will serve as a reminder for the user regarding the percentage of each plant that should be consumed yearly. It's important to note that plants will be planted on different levels of soil, where ground rated 2 to 1-star will produce much less energy compared to 3-star ground. This is due to the fact that less sunlight leads to reduced photosynthesis and calorie production.

Users can experiment with different types of plants using the simulator, which currently offers ten choices. Additionally, they can adjust the number of plants they wish to grow each year based on the simulator's generated results. Furthermore, users can select three types of crops and specify the proportion in which they would like to grow them. For instance, if the user prefers beans over rice, they can adjust the proportions accordingly. Alternatively, if there is a specific crop that is easier to cultivate or trade, users can incorporate this into their final arrangement.

# **8 Planting Crops**

Once all inputs have been entered, the Space Data Generator will automatically calculate the user's plant consumption and provide a breakdown based on percentage. Any surplus production will be quantified in US dollars. In this scenario, the user should strive to maximize profits since all their consumption needs have been met. Therefore, careful consideration should be given to how to achieve the most profitable outcome.

### **9 Display**

Then LDT toll will show the final room, routine, solar panel, and crop location.

#### **10 Result**

The data produced by this tool has the potential to benefit various scenarios. For instance, it could be transformed into an agricultural product or planting tool, particularly in situations where multiple types of plants need to be integrated. In order to maximize the tool's productivity, we are particularly interested in determining the optimal location for the solar panel. This will help to demonstrate the effectiveness of the tool in practice.

### **11 Example**

Based on our analysis, we have chosen to conduct a field test of our tool in Guangdong province, specifically on a site with an expansive area of 250,000 square meters. The site includes a small forest and existing buildings, and the average household electricity consumption in the area is estimated at 10,715 kWh. Our primary requirement is to identify a sun-efficient location for the placement of the solar panels. Subsequently, roads will be constructed towards the four corners of the site to facilitate access.

In the next step of our plan, we aim to strategically position the solar panels in the most optimal area of the site, which has been rated with a three-star energy efficiency rating based on power consumption data. The energy transformation rate of the solar panels is assumed to be 15 percent, and the total energy required to meet our goals is estimated at 72,000 kWh. To achieve this, we will be utilizing a genetic algorithm, which is a computational optimization technique that can identify the optimal location for the placement of solar panels on a 100-square-meter land area, taking into consideration the three-star energy efficiency rating. This approach will enable us to maximize the energy generation potential of the solar panels and efficiently meet our energy production target (Fig. 2).

Based on the given scenario, the household's main dietary staple will be rice, with other nutritional requirements being met based on the household's needs. Considering a household size of ten people, the estimated daily consumption of rice would be approximately 9000 cal. The position of the plant will be placed by pixel farming [8], assuming an energy transformation rate of around 1 percent for rice cultivation, the total land energy required for rice production would be approximately 900,000 cal, equivalent to 1000 kilowatt-hours (KW) of energy. This would require approximately 800 square meters of land for cultivation (Fig. 3).

The remaining land can be utilized for cultivating the most cost-effective plant based on market value.

**Fig. 2.** Top image shows how to calculate the housing, and solar panel location. (by Wang Yueyang)

**Fig. 3.** Final result (by Wang Yueyang)

# **12 Conclusions**

Space data generator has successfully showcased our design aspirations as a whole. This tool has the potential to evolve into an optimized app that can automatically access realtime information, such as global crop prices and the comprehensive impact of soil quality on different crops, to quickly provide universal carbon net zero emission principles [9]. Integration with pixel farming could significantly enhance the efficiency of addressing agricultural planting challenges in regions with low birth rates. This app has the potential to greatly assist farmers worldwide in an intuitive and user-friendly manner. However, researchers can further research and explore how the Space Data Generator can consider different climates, such as in frigid zones and tropical regions, where users' lighting needs may vary. Additionally, Users can enhance the pedestrian circulation and housing design by providing better connectivity between the architecture and essential locations on the site [10] (Fig. 4).

One potential area of improvement in agriculture is the adoption of agroecological practices, such as mixed planting or intercropping, where different crops are planted together to meet diverse agricultural needs. This approach has the potential to enhance eco-efficiency as mixed planting requires less fertilizer and promotes natural pest control. Additionally, mixed planting can improve soil health and biodiversity. Furthermore, this design approach can also be applied in analyzing optimal urban planning strategies, considering parameters such as transportation, economy, visual aesthetics, and energy consumption for lighting. By assigning appropriate weights to these parameters, this approach can serve as a valuable tool in urban design. In conclusion, to achieve greater efficiency and compatibility, developers need to enhance the Space Data Generator's functionality through continued editing and refinement. Regardless of how users use this tool, with further improvements, it can deliver even more efficient functions in the future.

**Fig. 4.** This image showcases the promising potential of the methodology employed by this app. (by Wang Yueyang)

# **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **The Use of Normative Energy Calculation for Natural Ventilation Performance-Driven Urban Block Morphology Generation**

Wenjing Li1, Xinhui Xu1, Mehdi Makvandi1, Zhuoyang Sun2, and Philip F. Yuan1(B)

<sup>1</sup> College of Architecture and Urban Planning, Tongji University, Shanghai, China Philipyuan007@tongji.edu.cn

<sup>2</sup> School of Architecture, Shanghai Institute of Technology, Shanghai, China

**Abstract.** Exploring the three-state coupling relationship between "urban block morphology, carbon emissions, and human comfort" is necessary when making preliminary design decisions. Currently, morphology generative design is subject to interactions between the level of model definition and simulation duration. Self-intelligent and intelligent generative design workflows using evolutionary algorithms are now becoming an effective solution to this problem. This paper incorporates a dedicated controllable ventilation model based on a normative performance calculator and proposes it in the morphology feedback generation execution of the automated design process. The aim is to develop this automated design method from ambient environment driving only to outside-interior coupling natural potential ventilation influencing morphology generation, with the aim of providing technical support for carbon emission performance-oriented and indoor human comfort-oriented design of urban blocks.

**Keywords:** Generative design · Urban block morphology · Quantification method · Natural ventilation

# **1 Introduction**

"Double carbon" drives spatial generative design, and digital empowers street-level smart growth. While meeting the people's growing demand for building comfort, controlling the growth of building energy consumption is one of the keys to achieving the "dual carbon" goal. Building operation energy consumption accounts for 22% of China's total social energy consumption, and construction energy consumption accounts for 11%. In the Reshaping Energy Scenario in 2060, the emission reduction potential of the construction industry is 74%, which is 1.5 times that of the industry, accounting for the largest proportion of the three energy consumption industries, and it will contribute approximately 50% to the peak of carbon emissions ahead of schedule. Energy savings [1]. Taking into account the impact of urban microclimate on the energy consumption of buildings, conducting research on space self-generating design methods is conducive to comprehensively weighing building energy consumption and digitally empowering urban smart growth. The fourth generation of urban design should aim at the theoretical reconstruction of morphological integrity and take the transformation of digital technology methods and tools of human-computer interaction as the core feature [2]. The expression paradigm of urban planning and architectural design based on computational design thinking is promoting the evolution of traditional space construction and environmental materials in the direction of "performance-driven form" intelligent design. Low-carbon urban planning at the macro level, sustainable block design at the meso level, and green building design at the micro level all contribute to the establishment of a resilient living environment. With the help of artificial intelligence tools, planners and architects can break through the shackles of this linear science prior to construction activities and Post-occupancy evaluations, the high point locates the urban space [3, 4].

#### **1.1 Core Algorithm of Carbon Emissions**

The core of calculating carbon emissions during the operational phase of a building complex is building energy consumption calculation. Based on the direction of data aggregation, existing Urban Building Energy Modelling (UBEM) algorithms for street-block building complexes can be divided into top-down and bottom-up approaches. The latter focuses on individual building energy consumption rather than overall regional building energy consumption modeling, which results in higher accuracy and facilitates the evaluation of the impact of new standards and technologies on building energy consumption in a "dual-carbon" direction, but is relatively time-consuming. The bottom-up approach can be divided into physical methods and data-driven methods, with physical methods further divided into detailed physical models and simplified engineering models. The scale of calculating street-block building complex energy consumption is much larger than that of single building energy consumption calculation. Compared with detailed physical models, engineering models are easier to program, operate more quickly, and can be better combined with data-driven models, embedded in building intelligent design workflows, and support "dual-carbon" driven spatial generation research [5].

It is generally believed that engineering models are inferior to physical detailed models in terms of computational accuracy due to the simplification of computational conditions. However, Godfried Augenbroe et al. [6–8] pointed out that some engineering models are not less accurate than detailed physical models when supporting building performance calculations in the planning and design stages, i.e., solving the Fuzzy Topology problem in the conceptual phase. In fact, they even demonstrate better robustness in comparative studies. There are two main reasons: (1) the design stage is a process where conditional information is constantly input, increasing the amount of graphic information and decreasing abstraction. The quantity, form, and quality of space are interdependent and restrictive, constantly changing, and the amount of quantifiable building information is limited. At this stage, a large number of key parameters must be assumed in order to carry out targeted modeling (Design Performance Modeling, DPM). Therefore, there is a possibility of calculation deviation due to insufficient assumptions. (2) Detailed physical models also require assumptions and simplifications, which can also increase calculation deviation. De Wit et al. conducted research on the impact of uncertainty factors caused by this simplification on building energy consumption simulation, proving that this impact cannot be ignored in most cases.


**Table 1.** Classification of core algorithms for urban building energy consumption

Hence, the proposed intelligent and shape-based workflow in this project will employ a "grey box model" (engineering model) rather than a "black box model" (data-driven model) during the critical phase of building performance calculation. This will enable the completion of building performance evaluation, encompassing the carbon emissions index as well.

#### **1.2 Carbon Emissions Calculation Platform**

The existing engineering models for energy consumption analysis of urban building groups are mainly based on the classic IS013790 algorithm. ISO13790:2008 provides 3 calculation methods for the design and evaluation of the thermal and energy performance of buildings with different degrees of complexity: monthly steady-state calculation method, simple hour-by-hour dynamic calculation method (quasistatic), and detailed dynamic simulation method [9]. The main building energy consumption calculation platforms based on this algorithm include SimStadt, a 3D urban energy platform developed based on the GIS database, which can provide energy consumption analysis of building stock or individual buildings. The 3D spatial urban energy consumption simulation method developed based on 3D urban morphology data can support the evaluation of solar energy potential and heating demand of residential buildings. Urban Energy Maps, a residential energy consumption and greenhouse gas emission calculation platform based on a GIS visualization module, can provide local governments with energy performance monitoring of urban buildings. These software programs do not support real-time calculation and editing.

The Energy Performance Calculator (EPC, ISO13790) software ecology based on the standardized calculation method of quasistatic building energy consumption provides a new way to calculate the carbon emissions of urban buildings in the solution design stage [10, 11]. Among them, the calibration add-in EPC\_Calibration Add-In, which is used to correct the difference between the assumed value and the real value of the parameter, solves the problem of limited building information in the early design stage. As shown in Fig. 1, the software ecosystem provides seven main energy consumption calculations for building operations (heating, cooling, humidification, lighting, pumps, fans, hot water; heating, cooling, humidifying, lighting, pumps, fan, domestic hot water) and is able to convert that into carbon emissions. Its computational accuracy has passed the dynamic simulation comparison experiments under different model information granularity. Among them, Mayuri [12] and Kokogiannakis G. [13] studied the commonality and characteristics of EPC and physical detailed models ESP-r, EnergyPlus, and IDA-ICE in the calculation of building operating energy consumption. Adrian C. et al. [14] studied the application range of different precision calculation methods and refined simulations based on ISO 13790 in heating and cooling energy consumption calculations. Qi L. et al. [15] studied the influence of uncertain factors on building performance calculation under different urban scales, and compared the accuracy of steady-state calculation and refined finite element numerical simulation. Research proves that EPC is as accurate as or better than physical detailed models during the conceptual design phase [10, 15]. In addition, some scholars have indirectly verified the applicability of software ecology by applying EPC to specific engineering problems. For example, Ji-Hyun Kim, etc. [16] EPCs were used for LEED-EAc1 scoring. Sang Hoon L. et al. [17] used EPC to evaluate the performance of residential buildings. Therefore, based on EPC software ecology, this project develops an efficient calculation method for the energy consumption of street buildings. Here, the quasistatic calculation method is mainly used.

**Fig. 1.** Schematic diagram of EPC [17, 18]

#### **1.3 Calculation Path of Carbon Emissions**

The difference between the research plans for carbon emission calculation mainly lies in the strategy selection of thermal zoning, heat transfer model and algorithm in the thermal simulation step. It is generally believed that for the severe and cold climate areas where heating is the mainstay and the urban building types are relatively simple (such as residential buildings), the steady-state heat transfer model can be used to calculate the heating energy consumption indicators of various benchmark buildings, and then according to the total amount of various types of buildings (the number of buildings or building area) is used to weight the estimated subitems and overall energy consumption; for cities with a large proportion of cooling demand, such as hot summer and cold winter, hot summer and warm winter, and the building types are diverse or functionally complex when it is higher, a multizone dynamic heat transfer model should be used [19].

Automated building thermal zoning algorithms such as Autozoner can achieve rapid thermal zoning of a large number of individual buildings in urban areas. However, this method cannot incorporate microclimate factors in the classification and characterization; that is, considering the local environmental impact of the city and the spatial morphological characteristics of the building itself, clustering or typological standard unit settings are performed. In this regard, the academic community has made cutting-edge explorations in classification optimization. Cambridge University's light and thermal (LT) method [20] pioneered the physical calculation of energy consumption and urban form analysis separately: based on the parameters obtained by the form analysis, the corresponding energy consumption values were extracted from the LT curve set database, and then the overall energy consumption of the region was calculated by the interpolation method. This method makes many simplified settings in terms of climate boundaries, equipment planning and so on. MIT's shoebox algorithm (Shoeboxer)[21] On the basis of LT, the calculation accuracy is improved, and single building (unit) partial clusters with similar thermal characteristics (function, orientation, external occlusion, etc.) are merged into large regions, and the energy consumption intensity of the corresponding representative unit modules (so-called "shoeboxes") in each region is calculated and weighted. The algorithm requires that the indoor temperature setting of each partition is not very different, and the clustering standard is set reasonably. Building Block Energy Estimation (BBEE) of Tsinghua University [22] It consists of two parts: a typical zone and an energy database. Among them, the typical thermal area is similar to the unit module in the shoebox algorithm; the energy consumption database is similar to the LT-Curves database in the LT method. The corresponding database lookup table is performed for the energy consumption of each typical thermal zone, and the weighted summation of the energy consumption of all zones is carried out. Georgia Tech's EPC ecology uses the LT method of separating the architect's work from the engineer's, continuing the precision advantage of Shoeboxer's physical simulation of typical thermal zones. The EPC software has a partitioned and hierarchical energy consumption calculation design [10, 23]. Furthermore, its regional energy performance calculation software (Network energy, NEP) takes into account the accuracy of energy consumption calculation in the case of energy supply interaction between regional buildings (such as energy allocation during peak energy consumption) [17]. Due to the limitations of time and energy, the interaction mechanism of energy supply among regional building groups is not within the scope of this paper.

Based on the digital workflow for building generation driven by outdoor wind environment material performance, this paper incorporates the developed normative building energy consumption calculation method ventilation module to explore the formation of a new workflow for building generation driven by the coupling of indoor and outdoor wind environment material performance.

# **2 Methodology**

### **2.1 Shape Path**

The overall research approach for the study of the morphology generation of urban center spaces with wind and environmental performance orientation through the coupling of form-carbon-human is shown in Figs. 2 and 3. In terms of "form", the aim is to control the parameters of building clusters under wind and environmental conditions. In terms of "carbon", the potential for natural ventilation energy saving during the operation stage of each generated building cluster is calculated. In terms of "human", outdoor pedestrian wind environment comfort and indoor comfort are taken into consideration. In order to study the central area of the coastal new city, machine learning is used to establish the street pattern model of the city, and natural ventilation-related variables and evaluation methods are systematically set in the development of the calculation model. This paper proposes the incorporation of a dedicated controllable ventilation model based on a normative performance calculator into the automated design process for morphology generation.

**Fig. 2.** Schematic diagram of the generative design method to be established

#### **2.2 Development of Efficient Computing Methods**

The research focus of this article is how to transmit the information of the outdoor wind information to the indoor environment. To this end, we opened an input value on the calculator with is detailed explain as follows, which is converted from the outdoor wind pressure and wind speed value. Then, indoor fresh airflow-related calculations are carried out, and finally, the delivered energy and carbon emissions are obtained.

Ventilation scene.

**Fig. 3.** Schematic diagram of "shape-carbon-human" coupling mechanism

*Hybrid ventilation scene*. In a hybrid ventilation (HV) scenario, natural ventilation is utilized to partially meet the cooling and fresh air requirements, based on a predefined outside air humidity threshold (either 70% RH or 80% RH). During every hour when the interior temperature (*Tin*) is higher than the ambient temperature (*Tam*) and cooling is required, the full cooling demand is met through the utilization of outside air. This approach is employed to determine the maximum potential of natural ventilation, assuming that the necessary air exchange can meet the cooling and fresh air requirements during these hours, even with a small temperature difference. However, outside of these hours, both cooling and fresh air are supplied by the mechanical system.

In this case, it is assumed that window opening control strategy is required in the simulation model. The reason behind this assumption is that during every hour when there is a cooling load and the interior temperature (*Tin*) is maintained at the set point temperature (*Tset*) by the mechanical cooling system, the complete cooling load is met through ventilation with outside air, provided the ambient temperature (*Tam*) is lower than *Tin*. This assumption is considered as the best-case scenario and referred to as the 'cooling potential.' However, achieving this cooling potential in the actual building depends on various factors, such as the window opening size and orientation relative to the prevailing wind direction, which are determined by the final design provisions.

In the case of active dynamic control of window opening strategy, the Energy Performance Calculator (EPC) includes this feature in the simulation model. An 'abstract' ventilation provision is assumed, where at full opening ratio, a ventilation flow of X air changes per hour (ACH) is achieved.

Our logic is as follows: if the interior temperature (*Tin*) is above the target and higher than the ambient temperature (*Tam*), we allow an appropriate amount of outside air to enter the building through partially open windows. In the PNV case, a minimum amount of fresh air is always supplied naturally. We have added this scenario to the EPC, and our control logic determines when windows should be closed or partially open. The window opening ratio (*Vratio* [m2]) is determined as follows:

$$V\_{ratio} = \begin{cases} C \ast ((T\_{in} - T\_{set})/(T\_{in} - T\_{am})), \text{ if } T\_{in} > T\_{am} \\ 0, otherwise \end{cases} \tag{1}$$

Here, *Tin* [°C] is the interior temperature, *Tam* [°C] is the ambient temperature, and *Tset* [°C] is the temperature set point (or rather, target point) for cooling. To avoid undercooling, particularly at night, we use Eq. (1) to gradually adjust the opening ratio and make optimal use of outside air while maintaining proximity to the target temperature. The amplification Factor C is introduced to regulate the speed of the controller's response to temperature differences.

The absolute PNV ratio (*Vabratio* [m2]) is derived according to Eq. (2), which simply guarantees that its value remains within the range of 0 to 1.

$$V\_{abratio} = \begin{cases} 1, \text{if} V\_{ratio} > 1 \\ 0, \text{if} V\_{ratio} < 0 \\ V\_{ratio}, \text{otherwise} \end{cases} \tag{2}$$

The flow volume of fresh air (*Vfresh* [m3/s]) is derived according to Eq. (3).

$$V\_{fresh} = \begin{cases} V\_{max} \* V\_{abratio}, \text{if} V\_{abratio} > 0.01\\ 0.01 \* \text{minimum}, \text{otherwise} \end{cases} \tag{3}$$

*Vmax* [m3/s] is the maximum airflow with fully open windows, as defined by the modeler. In theory, differences in the value of X will have only a small effect, as long as X is greater than 10. When the windows are fully open, the room temperature will quickly follow the outside temperature. Maximum air flow rate *Vmax*, as well as amplification factor C could be adjusted according to specific circumstances, if necessary, by using adaptive parameters, such as using machine learning algorithms to automatically determine the C and the *Vmax* based on historical data.

(*Vratio* \* *Vmax*)\*h\* ρ \* cp \* (*Tin* − *Tam)(*4*)*

*Qcool* represents the energy required for cooling. Air capacity represents the cooling capacity of air per unit time. The air capacity is represented by the product of the heat transfer coefficient h per unit area, the density ρ of air, and the specific heat capacity cp.

Furthermore, we would like to discuss the opening activity(*Vopen*) and position(*Vposition*) of windows on building facades. Firstly, building designs driven by natural ventilation performance as one of the main factors may not allow for the possibility of hinged windows (including those opened by occupants). It is generally assumed that the target building is equipped with a self-control system for windows. The term "windows" refers to the windows and their variations that face the outside of the building, such as the linear opening integrated into the window frame proposed by Godfried Augenbroe et al. [24]. These buildings use natural ventilation as a cooling source instead of mechanical ventilation, partially or completely, through the self-control system for windows, based on a whole-air conditioning system. This achieves energy savings without compromising thermal comfort. Secondly, when considering the position of windows, safety, and comfort in outdoor wind environments are generally the main factors considered based on regional building regulations. For example, Philip F. Yuan et al. [25] used factors such as average wind speed at pedestrian height, the ratio of comfortable wind speed at measurement points, wind speed dispersion, the ratio of calm areas, and the ratio of strong wind areas to generate the morphology of building clusters.

*Evaluation measures*

The following criteria were used to evaluate the HV performance of a building:

*Measure HV-1:* The factor of available hours for free cooling (*Rhour*) is derived according to Eq. (5).

$$R\_{hour} = \frac{H\_{\text{vent}}}{H\_{\text{nowent}}} \tag{5}$$

*Hvent* [h] is the annual total number of hours when the cooling need is completely covered by means of outside air, and *Hnovent* [h] is the annual total number of hours when the cooling need cannot be covered by means of outside air.

Measure HV-2: The factor of the cooling load reduction (*Rcooling*) is derived according to Eq. (6).

$$R\_{cooling} = \frac{E\_{\text{nvent}} - E\_{\text{vent}}}{E\_{\text{nvent}}} \tag{6}$$

*Event* [*kWh/m*2] is the annual cooling energy need with the use of natural ventilation, and *Enovent* [*kWh/m*2] is the annual cooling energy need without the use of natural ventilation.

#### **3 Results**

By displaying the carbon emission results with or without natural ventilation in the result input column, the natural ventilation potential is displayed (Figs. 4 and 5). Furthermore, this potential is used as a reward value to participate in iterative morphing. The main changes in the design process related to the new incorporated ventilation model are as follows.

**Fig. 4.** Building cooling carbon emission

#### (1) **Establishment of the Parametric Model of the Street Profile Space Building Environment.**

Typical block model refinement. Induction of different types of typical city blocks for random sampling and construction of a numerical simulation database. Through manual discrimination, partial clusters with similar thermal characteristics (function, orientation,

**Fig. 5.** Building operation and carbon emission

external occlusion, etc.) of the individual buildings in the prototype block are merged into a large area. For these large area types, preset the corresponding EPC (quasistatic high-efficiency calculation mathematical model for the carbon reduction potential of natural ventilation in buildings) input parameters (Fig. 6), and establish a corresponding database set There are two main types of parameters: I related to the indoor comfort maintenance of a single building and II related to the external protective structure of a single building.

**Fig. 6.** Parameter setting of thermal zones in a large area

# (2) **Construction of a mathematical model for the comprehensive evaluation of the carbon reduction potential of natural ventilation in buildings.**

Mainly through formula derivation and computational fluid dynamics simulation comparison experiments, the sensitivity parameters of the natural ventilation potential of different types of buildings are clarified; through mathematical model construction, the relationship between the sensitivity parameters of the natural ventilation potential and the corresponding performance indicators is defined; through the research and development of efficient calculation methods for the natural ventilation potential, a collaborative design platform for building performance evaluation is built (Fig. 7).

**Fig. 7.** Basic framework of comprehensive evaluation mathematical model

Development of an efficient calculation method for natural ventilation potential. The EPC building performance evaluation advantage platform is used to embed the control logic related to the sensitivity parameters of building natural ventilation potential and develop an efficient calculation program EPC\_v\_H2.0:NV.


(a) Mechanical parameters for each shape scheme of the agent processing.

Through the establishment of input parameters, the translation of relevant outdoor environment information to the indoor environment is completed. There are two main types of parameters: I is related to the general situation of a single building, and II is related to the thermal calculation of a single building.

(b) Wind environment data for each shape scheme of the agent processing.

Through the establishment of input parameters, the translation of relevant outdoor environment information to the indoor environment is completed. There are two main types of parameter sets: I related to the outer protective structure of a single building and II related to the climate information of the location.

#### (3) **Genetic algorithm-driven iterative optimization and application testing.**

The values of the design parameters are controlled by a genetic algorithm. The following two indicators are set as the control parameters at different levels of the optimization target, and the iterative optimization calculation is performed to obtain the optimal solution of the target building space design.


Examine the effectiveness of the EPC\_v\_H2.0: NV system in assisting design, especially how the system can provide architects with "task-specific" (task-specific) under different design conditions (site, building type) and different optimization goals. Specific) the feasibility of optimizing the results.

# **4 Conclusion**

Timely feedback of relevant information to architects and engineers during the primary design phase is crucial to overcome the limitations of empirical knowledge and achieve a performance-informed and performance-aware design process. Design Performance Modelling (DPM) facilitates rational dialogue, and this case study utilizes a normative energy calculation method, the EPC, to go beyond the performance rating of building design strategies as the context and purpose of the dialogue are constantly changing. This paper introduces a dedicated controllable ventilation model based on a normative performance calculator and proposes it in the morphology feedback generation execution of the automated design process. The aim is to develop an automated design method that goes from ambient environment driving only to outside-interior coupling, natural potential ventilation influencing morphology generation. This will provide technical support for carbon emission performance-oriented and indoor human comfort-oriented design of urban blocks.

Architecture-related studios and firms, such as the Architectural Intelligence Group (AIG), Digital Future Studio (DF), and AECOM iLAB (Innovation Laboratory), have shifted their focus to the primary design stage in response to the expectations expressed by owners and occupants, and their fulfillment by designers and building operators. Some of these firms even attempt to replace the work of designers by utilizing machine learning for intelligent management of design elements, architectural drawing recognition, and generative design for spatial form. However, building performance-driven design should strengthen human capacity, meaning that it should not only serve as an intuitive ruler to quantify design but also broaden designers' sense organs to provide better value. In other words, we advocate that architects should possess programming knowledge to achieve the most appropriate performance calculation and expand their design thinking to the field of software creation. Providing this support is essential to avoid limiting the human mindset with software constraints.

# **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Materialization and Construction**

# **Gesture Recognition for Feedback Based Mixed Reality and Robotic Fabrication: A Case Study of the UnLog Tower**

Alexander Htet Kyaw(B) , Lawson Spencer, Sasa Zivkovic, and Leslie Lok(B)

Department of Architecture, Cornell University, Sibley Hall, Ithaca, NY 14850, USA {hk385,wll36}@cornell.edu

**Abstract.** Mixed Reality (MR) platforms enable users to interact with threedimensional holographic instructions during the assembly and fabrication of highly custom and parametric architectural constructions without the necessity of two-dimensional drawings. Previous MR fabrication projects have primarily relied on digital menus and custom buttons as the interface for user interaction with the MR environment. Despite this approach being widely adopted, it is limited in its ability to allow for direct human interaction with physical objects to modify fabrication instructions within the MR environment. This research integrates user interactions with physical objects through real-time gesture recognition as input to modify, update or generate new digital information enabling reciprocal stimuli between the physical and the virtual environment. Consequently, the digital environment is generative of the user's provided interaction with physical objects to allow seamless feedback in the fabrication process. This research investigates gesture recognition for feedback-based MR workflows for robotic fabrication, human assembly, and quality control in the construction of the *UnLog Tower*.

**Keywords:** Mixed Reality · Gesture Tracking · Feedback Based Fabrication · Robotic Fabrication · Object Detection · Quality Control · Human Computer Interaction · Human Robot Collaboration

# **1 Introduction**

Since the mid-90s, Virtual Reality (VR) and Augmented Reality (AR) have existed under the umbrella of Mixed Reality (MR) on the Reality-Virtual (RV) Continuum between the absolute real environment and the absolute virtual environment (1). As VR and AR 3D user interfaces (3DUIs) have continued to become ubiquitous in architecture, construction, and academic research, the Milgram and Kashino's definition of MR has been further refined (2). In recent research, MR is often cited as an environment-aware overlay of digital content in the physical world where users are able to interact with both the digital and physical simultaneously (3). To facilitate human interaction between the digital and the physical MR environments, MR systems employ various techniques for collecting environmental and human physiological data, such as spatial mapping, hand-tracking, eye-tracking, and auditory recording. MR-enabled devices, such as the *Microsoft HoloLens 2* and *Meta Quest Pro*, utilize sensors, mics, and cameras to capture real-time data on changes in user behavior and the physical environment (4).

During the last decade, research using AR and MR workflows in the area of architectural fabrication have increased exponentially (5). Projects such as *Woven Steel*, *Timber De-Standardized*, *Code-Bothy*, and many more have explored human interaction with digital instructions in MR through digital interfaces such as buttons and menus or fiducial markers such as QR codes and AruCo markers (6–8). These MR fabrication projects have primarily focused on using human interactions with digital interfaces as the primary means to update the 3DUIs with new information. However, there exists an opportunity to directly incorporate human interaction with physical objects to update the 3DUI without needing digital interfaces.

This research integrates human interactions with physical objects through real-time gesture recognition as input to modify and update information in the digital environment. Through gesture recognition, touching a physical object could modify, update, or generate new digital information enabling seamless stimuli between the physical and the digital world. By recording user gestures as they interact with physical objects, the three-dimensional user interface can automatically provide new information in real time. As a result, the digital environment is generative of the user's provided interaction with physical objects. Through gestural tracking, user interactions with physical objects are recorded to determine the real-time location of physical objects in the digital environment. This can generate information such as localizing robotic tool paths, recognizing components, or measuring inaccuracies between the physical and the digital model. The real time generative data in the MR 3DUI allows the user to quickly respond to previous actions. The real time, feedback-based MR environment represents a cybernetic system whereby the outcome of interacting with a physical object(s) is taken as input for further action, thus creating a feedback loop until a desired condition is achieved.

The relationship between MR, gestural movement, digital twin, cybernetics, and human-computer interaction are used to help define systems of interaction between user and machine. From these relationships, the research presents three distinct *Gesture-Based Mixed Reality* (GBMR) fabrication workflows; (a) *object localization*—registers the location of a physical object in the digital space, (b) *object identification*—differentiates physical components using their digital parameters, (c) *object calibration* measures discrepancies between the physical object and associated digital geometry. Each of these three methods were used in six different tasks to construct the *UnLog Tower* (Fig. 1). The workflows derivative of this research presents new opportunities for human-machine co-creation within physical and digital environments through MR in architecture and fabrication industries.

# **2 State of the Art**

Innovative fabrication research projects such as *Holographic Construction*, *Code-Bothy*, *Woven Steel*, *Bent*, and *Timber De-Standardized 2.0*, use interactive "buttons" for users to toggle between different sets of digital geometry which is visible in the 3DUI (6–10). Though each of these projects use a *Microsoft Hololens* with *Fologram's* plug-in for *Rhino3d* and *Grasshopper*, the "buttons" can equally be interacted with one's mobile

**Fig. 1.** The UnLog Tower, photo by Cynthia Kuo.

device. In each of these precedents, the "button" is a custom, pre-defined clickable digital object (either mesh or poly-surface). Thereby any change in the virtual interface is dependent on the user interacting with the select, pre-defined "buttons" or otherwise manipulating other digital geometry. *Holographic Construction* and *Code-Bothy* use digital "buttons" to toggle up and down between rows of bricks as they are laid (8, 9). *Code-Bothy* has the added effect of color coordinating the amount of rotation per brick (8). *Woven Steel* and *Bent* exhibited several buttons to aid in the complex bending of tube steel and sheet metal (6, 10). *Timber De-Standarized 2.0* developed menu list to visualize different aspects of an inventory of scanned irregular log meshes as well as cataloging and designing with the members through operations of slicing, indexing, baking, and isolating (7). Though these precedents offer an interaction between the user and the digital geometry, the interactions are limited to digital menus and buttons.

Other research projects such as *Augmented Feedback*, *Timber De-Standardized 1.0*, and *Augmented Vision* use various methods of AruCo markers for tracking, physics simulation, and real-time scanning to create an active responsive environment between digital and physical objects (11–13). In *Augmented Feedback,* AruCo makers were placed at nodal intersections of a bending-active bamboo grid-shell structure (11). AruCo marker tracking allowed users to digitize the locations of the markers and provide graphic feedback for all active users through the head mounted display (HMD). *Timber De-Standardized 1.0* utilized a physics simulation for fabricators to visualize and virtually "drop" irregular scanned meshes of logs till they found their resting point, which allowed for a precise alignment with its associated physical log (12). Finally, *Augmented Vision* uses the *Hololens 2* to track and scan the user's environment then display such information to inform the progress of constructing a minimal surface with strips of paper and/or bark (13). These projects have demonstrated the capabilities of feedback-based MR using additional systems such as AruCo markers, scanned meshes, and simulation.

Additionally, the accuracy of AR/MR platforms presents a significant challenge in many of these AR/MR fabrication workflows. The accuracy of the fabrication instructions provided to users depends on the precision of the system. As a result, several studies have been conducted to assess the accuracy of AR/MR systems. Researchers have investigated the use of AR for assembling metal pipes (14), weaving bamboo structures (15), and constructing complex wall systems with bricks within a tolerance of 20 mm (16). Moreover, there have been research efforts aimed at improving the accuracy of AR/MR systems. A recent study by the authors explored the use of multiple QR codes to achieve a tolerance below 2mm with the Microsoft *HoloLens* 2 (17). The results of this study indicate that AR/MR systems have the potential to be used for high precision applications, such as assisting in robotic fabrication and accurate quality control.

# **3 Aim and Objectives**

While previous MR projects have focused on using menus, AruCo markers, scanned meshes, and simulations to interact with digital geometries, this project investigates the potential of incorporating user's tactile interaction with physical objects as an input to update the 3DUI. Enabled by gesture recognition, this research demonstrates new methods to use both digital and physical stimuli for a generative MR fabrication experience. This research has developed 6 experiments to test 3 GBMR fabrication workflows for tasks such as generating geometry relative to physical objects, localizing robotic tool paths, recognizing discrete components according to parameters such as height and length, or measuring inaccuracies between the physical and the digital models. The methods for this research will first present the tools and software to conduct this research, which will then be followed by the three GBMR workflows used to fabricate the *UnLog Tower*: (a) *object localization,* (b) *object identification,* and (c) *object calibration. Object localization* was used to determine the log geometry work object and the toolpath placement for robotic fabrication (Method 4.1). *Object identification* is utilized to identify physical components and display intuitive step-by-step assembly instructions (Method 4.2). *Object calibration* is employed to ensure the adjustment of jigs and the connection of panels match the digital model (Method 4.3). Each of these workflows will demonstrate new methods in MR research whereby physical stimuli can become a generative tool to interact and inform MR fabrication in real-time.

# **4 Methods**

The following studies were conducted using Microsoft *HoloLens 2* and *Fologram*, a AR/MR plug-in for *Rhino3D* and *Grasshopper* (18–20). The near depth sensing camera on the Microsoft *HoloLens 2* is used for articulated hand tracking (AHAT). AHAT tracks the movement and gestures of the user's hand, independent from the visible light cameras (VLCs) used for simultaneous locating and mapping (SLAM). The articulated hand tracking system recognizes and records twenty-five 3D joint positions and rotations, including the wrist, metacarpals, proximal, distal, and fingertip joints (21). This data is live streamed from the *HoloLens 2* device to *Rhino3D* and *Grasshopper* via *Wi-Fi*. The Microsoft AHAT API provides access to the built-in gestural recognition algorithm of the *HoloLens 2*, enabling the utilization of its advanced capabilities for hand tracking purposes. The joint configuration and orientation obtained from AHAT can facilitate the estimation of hand poses, such as pinching, tapping, or poking (22). This study focuses on the use of pinching as the primary mode of gestural interaction by the user. The pinching gesture is recognized when the thumb tip and index fingertip are in close proximity (Fig. 2). Additionally, a device capable of AHAT programming is imperative for gesture recognition and therefore is integral to the GBMR workflows.

**Fig. 2.** Digital twin of *HoloLens 2* headset location, joint configuration, and orientation from AHAT (Articulated Hand Tracking); visualized through headset (left); visualized through*Rhino3D* and *Grasshopper* (right)*.*

#### **4.1 Object Localization**

The *UnLog Tower* exhibits robotically kerfed timber round woods that have been stretched along two threaded rods to form panels through a similar method exhibited at the *UnLog* pavilion at University of Virginia (23). Logs are irregular geometries that are comprised of knots and sometimes curved but can nonetheless be abstracted to a cylinder in most cases. Before the log is robotically kerfed, it is cut in half. To localize the robot targets to cut the log in half using a 6-axis robotic arm with a 5hp bandsaw end-effector, *object localization* method was employed. The user would place three points at both ends of the log to create two individual circles to generate a cylindrical mesh that was in line with the physical log (Fig. 3). Each point was created by the user pinching their right-hand index finger to their thumb. This feedback mechanism provides the user with a visual confirmation of the digitization process. From the cylindrical mesh, a surface was generated in the middle of the cylinder whereby the robot tool path could be derived from the robot targets at either end of the surface using *Robot Components* (24)*,* a robot programming plug-in for ABB robots in *Grasshopper* that is then copied into *Robot Studio,* an ABB software for programming ABB robots (25).

**Fig. 3.** *Object localization* is used to generate the location of a cylinder according to the diameter(s) of the log to automate the placement of the robotic toolpaths.

Once the log is cut in half, one half of the log is rotated 90° and remounted in the robot cell. According to the structural requirements for the *UnLog Tower* the cross section of each board was to be no less than 5- by 0.75--. For each half log, the top and bottom ends of the log were to be trimmed off. The fabricator was to check the location of the cut surfaces within the log to ensure that the boards would meet the minimum cross-sectional requirements without any of the cut surfaces colliding with the 4- x 4-- log mounts (Fig. 4). Figure 4 demonstrates the process whereby the user can locate the half log in the robot cell by placing three points; two at one side of the half log to determine the diameter and one at the opposite end to determine the length of the half log. After the log geometry is defined, the user can set the location of the cut geometry by placing a point on the profile of the log (Fig. 5). The MR system offers the user ongoing feedback during the process by performing a validation to determine whether the cut geometry falls within the boundary of the log. In the event that the cut geometry is placed outside the log or is situated too close to the log mount, a red notation with a cross mark is displayed within the 3DUI. The user may then respond to the alert and adjust the location of the cut geometry until a satisfactory outcome is achieved, represented by a green notation. The *object localization* workflow allows users to define points in the digital space that represents the physical log stock for work-object localization during robotic fabrication (Fig. 6).

#### **4.2 Object Identification**

*Object identification* is used to differentiate between self-similar physical components and display intuitive step-by-step assembly instructions. After the half logs have been robotically kerfed, they are set aside and prepared for finger jointing. The finger joint template not only includes an outline for the finger joints, but also an outline for the hole that the threaded rod will ultimately pass through. Because of the parametric design of the kerfed timber panels for the *UnLog Tower,* the finger joint locations are staggard

**Fig. 4.** *Object localization* is used to determine the work object placement for robotic fabrication.

**Fig. 5.** *Object localization* is used to determine the placement of the toolpath for robotic fabrication.

**Fig. 6.** *Object localization* system diagram describing how user interactions physical objects are used to create digital data through gestural recognition.

between adjacent boards within each half log. In order to correctly mark the location of the finger joints and the location for the threaded rod holes, GBMR was employed for *object identification* to correctly situate the location of the template per each board layer by registering the distance from the top of the board to the ground (determined by the QR code placement). The system determines the corresponding virtual template to display by comparing the calculated distance between the user-defined point to the ground with the predetermined distances of the virtual templates to the ground. The virtual template has an added notation that tells the user which layer they are on, so that the user can be sure that the physical template is being placed appropriately (Fig. 7). The finger joints were cut with an oscillating saw and drill, while the holes for the threaded rods were drilled with a hole saw (Fig. 8).

**Fig. 7.** *Object identification* is utilized to identify physical components and display intuitive step-by-step assembly instructions.

**Fig. 8.** Robotically Kerfed logs with finger joints and threaded rod holes

Additionally, *object identification* can be used to index and coordinate between selfsimilar parts. In order to brace the kerfed wood panels, the interior of the *UnLog Tower* exhibits 3 unique reciprocally steel tube frames. There are 9 unique tube lengths amongst 54 total steel tubes (Fig. 9). After the steel tubes were cut to length, *object identification* was employed to index the tube steel according to their length and communicate to the user the location of the tube steel in the digital model(s) (Fig. 10). By placing a point at either end of the of the tube steel through gesture recognition, the user can define the length of the tube steel, which is checked against a list of tube steel lengths predetermined in the digital model. If the value between the user defined length and a predefined length is within a set tolerance of 0.5 inches, the 3DUI displays the corresponding digital information to the user through notation and two coordination models that visually indicate the location of the tube steel in the overall structure. The coordination model on the left (Fig. 10b, c) illustrates at 1:1 scale the tube steel location within a particular tube steel frame and the coordination model on the right (Fig. 10a–c) illustrates at 1:10 scale a virtual model of the *UnLog Tower* with the location of the tube steel within the whole model. By using predetermined distances and gestural recognition, *Object Identification* can be used to pair digital assembly instructions with the identified physical object (Fig. 11).

**Fig. 9.** Reciprocally framed tube steel in the *UnLog Tower,* photo by Cynthia Kuo.

**Fig. 10.** *Object identification* is utilized to identify physical components and display part to whole assembly instructions.

**Fig. 11.** *Object identification* system diagram describing how digital assembly is filtered through object identification via gestural recognition.

#### **4.3 Object Calibration**

In order for the kerfed logs to splay out into panels, the threaded rods needed to have pre-located hex nuts appropriately placed to ensure that each board member would be in the correct location. In the GBMR workflow, *object calibration* was employed to place the hex nut locator correctly along a plywood jig. The hex nut locator was 3D printed with PLA to firmly hold each hex nut when it was screwed into the plywood board. A digital twin was created for each hex nut locator. When the user pinched the corner of the locator, *object calibration* would use gesture recognition to continuously track this movement, thereby synchronizing the digital geometry with the physical. As the physical object moved closer to the goal position, the notation would transform from red to yellow to green once the physical was properly located (Fig. 12). This workflow represents a cybernetic system in which the adjustment of the physical locator position will generate new virtual feedback for the user, thus creating a feedback loop until the desired condition is attained. The desired condition is achieved when the digitized physical location of the hex nut locator is within a tolerance of 0.125 inches. This is indicated to the user via the notation system where the red or yellow cross turns into a green tick. The MR system will instruct the user to move onto the next hex nut locator only after the previous hex nut locator is correctly placed via gesture recognition. After all the hex nut locators were properly placed, a threaded rod is screwed through jig (Fig. 13).

**Fig. 12.** *Object calibration* is employed to ensure the hex nut locators are adjusted to match the digital model. As the physical hex nut locator moves closer to its digital position, the notation would transform from red to yellow to green.

**Fig. 13.** After all the hex nut locators were properly placed, a threaded rod is screwed through jig

For the panel assembly, the robotically kerfed logs were splayed out along two threaded rods with pre-located hex nuts as was done in the *UnLog* pavilion (23). Temporary custom slip washers were placed between the hex nut and the board to ensure that the boards would keep their position until joined into larger prefab components with steel slip washers. Once panels were joined together in larger prefab components, *object calibration* was used to check the location of each board as they were tightened into location (Fig. 14). This quality control step aligned a digital model of the goal geometry to the physical panel using the placement of a QR code. The physical location of the boards was determined by using GBMR to place a point at the center of the finger joint location of each board, which was automatically checked against the closest digital board. The deviation between the digitized board location and the digital board allowed for a 0.125- tolerance. A red cross notation indicates if the deviation was outside the tolerance, otherwise a green check notation would appear. This quality control step ensured that the parametrically defined wall panels were properly calibrated into larger prefab wall elements that were then transported to the site for assembly (Fig. 15). By using the distance between physical and the digital object as variable, visual feedback is provided to the user during fabrication (Fig. 16).

**Fig. 14.** *Object calibration* is employed for quality control of prefab wall components.

**Fig. 15.** Details of the *UnLog Tower*: finger joint splice connection (left) and robotically kerfed logs stretched along a thread rod (right), photos by Cynthia Kuo..

**Fig. 16.** *Object calibration* is employed to ensure the adjustment of jigs and the connection of panels match the digital model.

# **5 Results and Discussion**

The implementation of gesture recognition for GBMR was incredibly useful for the fabrication of irregular and parametrically defined building components exhibited in the construction of the *UnLog Tower* (Fig. 15). The prefab wall panels were attached to the tube steel reciprocal frames on site and lifted onto the foundation with a boom forklift.

The implementation of gesture recognition in MR fabrication workflows allowed users to define physical objects without the arduous placement of AruCo markers. The *object localization workflow* demonstrates that gesture recognition can be employed to locate robot work object data (Fig. 6). However, the utilization of gesture recognition assumes a certain level of dexterity on the part of the user, as the data is dependent on the fidelity and accuracy of the user's fingers. The object localization workflow can be modified for robotic fabrication procedures that require a higher tolerance. Alternatively, improvements in the AHAT, articulated hand tracking, on the *Microsoft HoloLens 2* would also increase the accuracy of the overall system and the resolution of the work object placement.

The research also describes the potential of using gestural tracking for *object identification* whereby the user's hands can be intuitively used to index and coordinate objects between self-similar parts based upon predefined parameters (Fig. 11). While this study utilizes the varying lengths of components as the parameter, future studies could begin incorporating the boundary geometry or volume in the workflow. This workflow holds enormous future potential for fabricators and programmers to develop future projects that employ this method to coordinate and educate subcontractors on the construction of parametric components with discretized or self-similar parts.

Finally, the *object calibration* workflow is a unique way for users to synchronize between physical objects and their digital twins (Fig. 16). The threaded rod test is unique in that the user can pinch the hex nut locator while moving the physical object. Conversely, the second test with the panel quality control demonstrated that some objects are too heavy or cumbersome to pinch while moving. For that reason, the second test demonstrated the use of gesture recognition to iteratively define critical points until the physical geometry aligned with the digital model. As is the case with the *object* *localization* workflow, the accuracy of the gesture recognition is limited to the user's finger precision. This method will have to be modified for higher tolerance fabrication projects. Additionally, the method could have been employed to locate the foundation steel on the existing concrete slab that was used to support the project.

# **6 Conclusion**

The future potential of using gesture recognition in MR fabrication projects is enormous. The presented research not only demonstrates that real time feedback through gesture recognition is imperative for advanced MR fabrication projects, but it can also be used in robotics, geometry creation, object indexing, model coordination, interactive digital twin, and complex quality control. Future investigations will seek to improve the accuracy of this method for high precision fabrication projects and explore the potential of incorporating a wider range of gestures, such as "tap", "poke", and "pinch". Additionally, a user-controlled interface is being developed to enable/disable or undo a recognized gesture.

The study highlights the potential of utilizing gestural recognition to innovate humanmachine fabrication processes. Through real-time gesture tracking, GBMR workflows can seamlessly blend real and virtual environments with visual feedback and tactile interaction. The three GBMR workflows exhibited in this paper demonstrate the various applications for the real-time feedback-based fabrication and assembly of the *UnLog Tower*. This phygital experience offers a whole series of future applications investigations in the field of Mixed Reality fabrication and Human-Machine co-creation.

# **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **A 'Human-In-The-Loop' Workflow for Realizing Taihu Rocks**

Zexin Chen2, Dandan Lin2, Lujie Sun2, Sining Wang1(B) , and Dongchen Han3

> <sup>1</sup> School of Architecture, Soochow University, Suzhou, China snwang@suda.edu.cn

<sup>2</sup> Research Center of Cultural Heritage Digitalization, Suzhou, China

<sup>3</sup> Suzhou Gold Mantis Construction Decoration Co, Ltd, Suzhou, China

**Abstract.** This research uses an expanded polystyrene (EPS) Taihu rock as a demonstrator to illustrate a workflow encompassing shape grammar-based design and Mixed Reality (MR)-aided robotic fabrication. It aims to address a postdigital mindset that values human's tacit knowledge and craftsmanship within CAD-CAM processes, therefore, this research combines three components: an idea of the human-cyber-physical system (HCPS), a from-finding approach, and an augmented materialization method. The investigators first 3D-scanned a natural Taihu rock and interpreted its geometric peculiarities into design generation rules. These rules were then translated into robotic foam-cutting paths. With Headmounted Display (HMD) and MR technology, human fabricators were able to alter robotic motions on-site per their aesthetical demands.

**Keywords:** HCPS · Mixed Reality · Robotic fabrication · Taihu rock

# **1 Background**

Today's architecture, construction, and engineering (ACE) industry have tremendously benefited from the development of computation and automation, urging the exploration of non-standard forms and material systems. Facing such complex design requirements, a fully autonomous CAD-CAM workflow remains a challenging mission insofar as it still demands human cognition and intervention in response to ill-defined construction tasks or material inconsistency. Hence, compared to highly automated design productions, a 'human-in-the-loop' system setup may increase workflow flexibility and efficiency [1]. Architectural discourses have entered a post-digital realm where the non-digital aspects started to regain importance and it aims at addressing the humanization of digital technologies [2]. Thus, the goal of this research is to investigate a 'human-in-the-loop' workflow that benefits from both the computation and machine capacity and the rich repertory of experience and knowledge from humans. To investigate this question, the research first introduces the idea of HCPS and based on which, it develops a system setup hinging on interactive human-machine collaborations. The research uses an EPS-made Taihu rock, as a demonstrator, to elaborate on the technical aspects. Here, the postdigital design and fabrication processes highlight bi-directional communications among humans, the computational model, and the industrial robot, which are made possible via parametric design and MR interface.

#### **1.1 Idea of HCPS**

As humans possess tacit knowledge vital to the cultural, political, and economic dimensions of their daily lives, the combination of which with machines' data processing capacity and high-level precision may alter the landscape of today's manufacturing industry [3]. Sowe et al. and Zhou et al. defined an adaptive human-centered project delivery system as HCPS, which values human control and communication peculiarities in the system so that real-time, efficient, and reliable data intelligence can be feasible [4, 5]. The idea of HCPS highlights 'human-in-the-loop'. It aims at achieving collective efforts by absorbing human's capacity of processing arbitrary information into digital systems, and by facilitating collaborative context-awareness based on data acquisition and status monitoring techniques [6]. The integrated system dedicates to augmenting humans for the sake of a dynamic interaction with machines, and the enrichment of human sensing and cognitive capabilities [7]. In this case, the idea allows the system to overlap the CAD realm with the CAM realm avoiding a predefined machine-centered 'workmanship of certainty' [8], instead, it enables humans to creatively intervene in the fabrication process according to their perception of aesthetics and ad-hoc decision-making.

#### **1.2 Shape Grammar-Based Design and Making**

Shape grammars have been identified with computational design and offer a natural basis for a computational theory of making. They use rule-based visual systems to describe and generate designs [9]. In today's digital practices, shape grammars can be parametrized using parametric design tools. Knight and Stiny referred to the idea of shape grammar-based making as 'doing and sensing with stuff to make things' [10]. Here, 'doing' and 'sensing' are interactively carried out by humans and machines, and based on what they extract from the elements, designers can define algebras for the 'stuff' of shapes and use these rules to make 'things'. For example, according to the analysis of the geometric rationale (sensing) of Gaudi's Sagrada Familia, today's digital designers have created parametric rules based on hyperbolic paraboloids and solid Boolean subtractions (stuff), then used these scripts to recreate the sculpture-like Passion Façade rose windows (things) [11]. Thus, shape grammar-based design and making can inspire human-machine collaborations as it ensures both rationality and openness at the same time. Such an idea can be found in architectural experiments such as 'Interactive robotic plastering' [8], 'RobotSculptor' [12], and 'RoMA' [13]. From different scales, these projects have demonstrated the technical feasibility of interactive human-robot collaborations and the possibility of real-time bi-directional communication between humans, cyber, and physical systems.

#### **1.3 Human-Guided Robotic Fabrication**

Human-guided robotic fabrication allows humans to interact with robots in real-time and to manipulate physical form during the fabrication process. Mitterberger et al. suggested such a semi-autonomous system setup features either manual handcraft with robotic precision or dynamic fabrication based on human-robot collaboration [12]. On one hand, handcrafting with robotic precision can establish a safer collaboration environment in which it requires devices to provide live feedback on design information and to help correct fabricators' operations. On the other hand, interactive fabrication allows the insertion of human creativity and aesthetics during the ongoing materialization process. Recent developments in machine vision, kinetic control, and MR have opened up new opportunities for 'human-in-the-loop' production, as human-guided robotic fabrication may offer a solution for ill-defined design requirements, complex material systems, and non-linear realization processes.

# **2 Method and Materials**

This research presents a 'human-in-the-loop' workflow that tackles the idea of HCPS and roots in the advances of shape grammar design and human-guided robotic fabrication. With parametric design and augmented reality technology, the system setup creates an open-ended design space allowing for humans' intuitive inputs and in-situ adjustments to the fabrication process. The research findings are highlighted in the realization of an EPS-made Taihu rock.

#### **2.1 System Setup**

The system setup, as shown in Fig. 1, consists of two major components: (1) a parametric associative model and (2) an MR-aided robotic fabrication platform. To implement shape grammar-based design, allow in-situ design and fabrication modification, and be used as Robot Operating System (ROS), the central associative model is created using Rhinoceros Grasshopper. The research started with extracting geometric information from natural Taihu rocks using a FARO Focus laser scanner, then the data was interpreted and transformed into the design generation tool via Grasshopper-based plug-in Wasp. The system adopts the MR application Fologram and HMD Microsoft HoloLens to establish bi-directional communications among the human fabricator, the design, and the fabrication platform. The robotic fabrication system includes an ABB IRB120 robot with customized foam-cutting end effectors and the live robotic control is enabled by Grasshopper-based plug-in Taco ABB. Also, to keep track of the ongoing fabrication, the investigators set up a stand-alone Microsoft Kinect for monitoring the as-built condition of the Taihu rock. Figure 1 elaborates on the system setup.

# **2.2 Form Generation**

Taihu rocks are common elements in private Chinese gardens. Their aesthetic characteristics can be generalized using 'slender, wrinkle, clear, leaky', as 'slender' represent the sculpture's odd shape with clear edges; 'wrinkle' represents concave and convex surfaces; 'clear' indicates the delicate carving from nature; and 'leaky' refers to perforations. Thus, the parametric interpretation of Taihu rocks will focus on geometric generation.

**Fig. 1.** System setup where A is the cyber system connecting three agents; B is the MR sys-tem; C is the robotic fabrication platform; D is the EPS unit; and E is the stand-alone moni-toring system.

#### **2.2.1 Data Collection and Interpretation**

The shape grammar-based design contains 3 steps (Fig. 2). Firstly, a FARO Focus laser scanner was used to acquire the geometric data of natural Taihu rock. It took 6 scans separated with a 60-degree angle between each other to rebuild a holistic virtual model. The retrieved point clouds have been converted to a continuous Mesh surface in Rhinoceros, which was then analyzed for its geometrical characteristics. In Fig. 3, the gradient colors indicate the undulation depth of the rocky surface, and the extracted information was categorized into structure factor and texture factor in order to interpret the Taihu rock's natural form.

**Fig. 2.** Design and materialization setup of a shape grammar formed Taihu rock.

#### **2.2.2 Shape Grammar-Based Design**

The investigators applied Boolean operation to recreate a target form based on the gained knowledge, they used cylinders, spheres, and ellipsoids to carve out the perforations, angularities, and surface undulations from cube aggregations. Wasp, a combinatorial tool in Grasshopper for designing discrete elements, was introduced for shape grammarbased parametric design. The plug-in can process a series of aggregations from the

**Fig. 3.** 3D scan of a natural Taihu rock and the analysis of its geometric characteristics.

input elements based on the pre-defined topological graph of connections. There were three steps in shape grammar-based design generation. The investigators first randomly placed connecting points on the surfaces of a single cube, defining a basic module for subtraction ('algebra' in Fig. 2). Then, the cube was further refined with Boolean operations using the aforementioned primitive geometries. Based on the cube aggregation and their connection points, the algorithm generated several cylinders and ellipsoids for further Boolean operations. Also, a boundary constraint was added to control the overall dimensions. A final design model was achieved through iterative Boolean operations with modules and primitive geometries.

In this case, the computationally generated form was for visual reference. To make grammars for design materialization, the subtraction logic was then transformed into robotic motion paths and corresponding carving tools. Based on Knight and Stiny's idea, the basic elements in shape grammar, "stuff", were replaced with customized tools. In terms of 'doing', the investigators rationalized the results from Boolean operations. To cope with the robotic end effectors including the how-wire cutter, L-shape, and U-shape carving tools, the perforations were transformed into hyperbolic paraboloids, and the undulation details were realized with 3D trajectories.

### **2.3 MR-Aided Robotic Fabrication**

A parametric associative model was created using Rhinoceros Grasshopper to bridge the virtual and the actual environment. With parameters that define the logic and shape of the Taihu rocks, the investigators were able to convert the design into instructions or trajectories for the later human-machine collaborative implementation based on material properties and site conditions. With the help of Fologram and Taco ABB, a tailor-made ROS based on Grasshopper, the human operators have been able to establish an MRaided robotic fabrication process that relied on human intuitions that allows for direct human intervention during the foam-cutting process.

#### **2.3.1 Fabrication Setup**

In this case, a marker-tracking method was used to overlay the digital model with the physical environment. The investigators placed a 20 \* 20 cm ArUco marker on the robot's fabrication platform and by fine-tuning the model's position in Fologram GUI, there were able to increase alignment accuracy.

The human-guided robotic fabrication contained rough cutting and fine carving. In the first stage, the fabricators removed large portions from the EPS cubes to shape a basic form of the Taihu rock. And in the fine carving stage, the investigators switched to U-shape and L-shape tools to add details of surface undulations and angularities. MR technology is the medium connecting humans, the cyber, and the physical system. It omits a great number of command inputs via keyboard and mouse, and the technology creates an intimate bond between humans (fabricators) and machines (robots). Such an HCPS setup was built on immersive yet natural human-machine interactions, and thanks to the off-the-peg applications available in Grasshopper, the investigators could realize a human-guided robotic fabrication workflow with a single manufacturing information model.

#### **2.3.2 Human-Guided Robotic Fabrication**

With the Grasshopper-based MR platform Fologram, the investigators have customized the holographic projection, including the cutting shape (hyperbolic paraboloid), control points of the central curve of the hyperbolic paraboloid (green spheres), and the robot's moving trajectory, in the Hololens operators' line of sight (Fig. 4). With Hololens' gesture tracking capacity, the investigators can interact directly with the digital model in a freehand manner. By switching on and off model layers, they were able to selectively choose the information needed for the fabrication tasks at hand and to avoid excessive information displayed on the screen.

**Fig. 4.** Human-guided robotic fabrication with customized tools. Rough cut with hyperbolic paraboloid (left) and fine carving with trajectories (right).

During the fabrication process, the investigators first properly position the EPS in the workspace according to holographic instructions. Then, based on the digital model generated from the shape grammar approach, they 'drag' the control points of the hyperbolic paraboloid per in-situ conditions. By combining Fologram and TacoABB, such manual modifications of the surface could be directly transformed into the updates of 'RAPID code' driving the IRB120 robot. Also, with Hololens, the investigators could have a visual preview of the robot's moving path, this helped to avoid potential human injuries caused by human-machine collision (Fig. 5). After the first stage of rough cutting with hot-wire cutter and hyperbolic paraboloids, the investigators changed the robotic end effectors into L-shaped and U-shaped tools for the next stage fine carving.

The fine carving operations led to the final features of the EPS-made Taihu rock. The initial robotic carving trajectories were inherited from the shape grammar-based design. Compared to the traditional robotic fabrication processes where the physical objects

**Fig. 5.** MR-aided robotic fabrication allows for a safe human-machine collaboration envi-ronment and flexible system setup.

normally remain stationary, the system setup in this experiment allowed the investigators to rotate the foam at will meanwhile accordingly altering the carving trajectories and the approaching angles. The investigators eventually carried out the fine carving operations in a sequence that required minimal machine calibrations.

The human-guided robotic fabrication also permitted the absorption of human creation and design input to the ongoing materialization process. By connecting the imagebased scanner Kinect with the parametric model through Grasshopper plug-in Project Owl, the investigators were able to track the geometric complexity of the Taihu rock in real time. Figure 6 illustrates the in-progress scan result of the EPS sculpture from different angles, where the color gradients indicate texture depths. Based on the available carving tools, in this case, it was the L-shape and U-shape stainless metal tubes, the investigators could improvise the actual carving outcome by manipulating robotic paths. For example, in order to create crumple moments in certain surface areas, they have overlaid several motion paths and tilted some of them to mimic natural appearances.

**Fig. 6.** In-progress scan using Kinect to evaluate geometric complexity.

# **3 Result and Conclusion**

In this case, the investigators used a small-scale EPS-made foam sculpture to demonstrate a workflow involving computational form generation and human-guided robotic fabrication. To accomplish this, the investigators developed a parametric associative model providing live holographic projections to human fabricators, tracking while translating human gestures into model inputs, and being used as the robotic operating system. By manipulating hyperbolic paraboloids and Boolean operations, the investigators were able to create complex geometric moments responding to the 'slender, wrinkle, clear, leaky' feature of the natural Taihu rocks. At the same time, the steady robotic motions resulted in smooth surfaces and continuous edges revealing a contrast to natural rocks, insofar as the aesthetics of the built outcome (Fig. 7) is the combination of digital cleanness and natural randomness.

**Fig. 7.** The built outcome of EPS-made Taihu Rock.

The 'human-in-the-loop' workflow demonstrated in this project can potentially benefit large-scale industrial designs and building practices. In this case, the result was limited by the robotic operating range, the type of end effector, and the material used. Future applications, however, may combine customized fabrication or construction robotics, tasks-oriented tools, and specific materials in order to have real-world meanings.

To conclude, the design-to-build experiment discussed in this paper aims to illustrate a 'human-in-the-loop' workflow that effectively combines human tacit knowledge, computation power, and robotic capacity. This design strategy enables humans to intervene creatively in the fabrication process, offering novel design-to-fabrication workflows within human-machine interaction systems. The system setup was designed to inherit the proportion and geometric peculiarities of natural Taihu rocks, a more intuitive and dynamic MR-assisted workflow could assist or replace the more conventional approaches, as well as to augment human craftsmanship and aesthetic sensitivity during an open-ended yet interactive fabrication process. With the emergence of HCPS in architecture, designers no longer adopt digital means for digital sake, instead, they are trending to seek the interplay between high-tech and high-touch experiences, between virtual and actual reality, between digital and analog material systems, wherein the objects they created can address humanization [2]. The experiment on the Taihu rocks can reflect an explorative human-machine collaboration that may be beneficial to real-world building assignments. By taking advantage of human's peculiar competence in handling ambiguous information and making judgment calls accordingly in most architectural intelligent manufacturing, the idea of 'human-in-the-loop' in architectural practice can offer the industry an effective approach for implementing digital twins and setting up a design solution space with appropriate tolerance.

**Acknowledgement.** This research was funded by the Social Science Foundation of Jiangsu Province (grant No. 21SHD001), National Key Research and Development Programmes of China (project No. 2022YFB4500601), and the Young Scientists Fund of the National Natural Science Foundation of China (project No. 52108022).

# **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Performance-Driven VR Learning for Robotics**

Shahin Vassigh, Biayna Bogosian(B) , and Eric Peterson

Florida International University, 11200 S.W. 8Th, St. Miami, FL, USA bbogosia@fiu.edu

**Abstract.** The building industry is facing environmental, technological, and economic challenges, placing significant pressure on preparing the workforce for Industry 4.0 needs. The fields of Architecture, Engineering, and Construction (AEC) are being reshaped by robotics technologies which demand new skills and creating disruptive change to job markets. Addressing the learning needs of AEC students, professionals, and industry workers is critical to ensuring the competitiveness of the future workforce. In recent years advancements in Information Technology, Augmented Reality (AR), Virtual Reality (VR), and Artificial Intelligence (AI) have led to new research and theories on virtual learning environments. In the AEC fields researchers are beginning to rethink current robotics training to counteract costly and resource-intensive in-person learning. However, much of this work has been focused on simulation physics and has yet to adequately address how to engage AEC learners with different learning abilities, styles, and diverse backgrounds. This paper presents the advantages and difficulties associated with using new technologies to develop virtual reality (VR) learning games for robotics. It describes an ongoing project for creating performance driven curriculum. Drawing on the Constructivist Learning Theory, the affordances of Adaptive Learning Systems, and data collection methods from the VR game environment, the project provides a customized and performance-oriented approach to carrying out practical robotics tasks in real-world scenarios.

**Keywords:** Robotics Training · Virtual Reality · Game-based Learning · Adaptive Learning Systems

# **1 Introduction**

The global economy is being rapidly transformed by sophisticated robots that enhance human dexterity, vision, speed, and strength. Meanwhile, advancements in automation technologies and Artificial Intelligence (AI) are changing how we view the future of work. These technologies are bringing disruptive changes to the employment market, requiring new skills and training in order to succeed in the future economy. Thus, preparing the workforce for an environment that is increasingly defined by these technologies is imperative.

Architecture, Engineering, and Construction (AEC) industries drive the design, construction, and operation of the built environment, affecting nearly every industry, trade, labor, and employment market across the global economy. However, despite their critical role in the job markets, the AEC industry is not ready to prepare their workforce for meeting the critical demands for a technically advanced workforce. Most buildings and infrastructure continue to be built on-site using centuries-old materials and techniques with labor construction productivity rates that are lower now than those reported over 50 years ago [11]. The training and apprenticeship model in the building trades has also remained largely restricted to on-the-job and on-site training with little emphasis on off-site and virtual training. Addressing the training needs of the AEC students and professionals is critical for ensuring the competitiveness of a large portion of the building industry workforce.

This paper outlines the process of developing an online robotics training system. The Robotics Academy, which is a game-based Virtual Reality (VR) learning platform, is designed to support the training needs of the AEC workforce. The Robotics Academy builds on research analyzing the AEC industry needs and leverages technological advances in adaptive, immersive, and data-driven systems to deliver a simulation platform for learning industrial robotics. Developing the platform draws on wellgrounded learning theories to provide an innovative, effective, and personalized training environment.

The research and design of the Robotic Academy have been supported by two grants from the National Science Foundation (NSF). The initial grant supported the efforts of an interdisciplinary team of faculty for creating a roadmap for the Robotics Academy and developing a prototype VR learning environment, which was completed in 2020. The second grant, which supports the project until 2025, builds on the prior work to expand the Robotics Academy by further research and development of the prototype and integration of AI into the platform.

In the following sections of this paper, we will describe the theoretical foundation for developing the Robotics Academy and the implications of advancements in AI on creating virtual learning environments. We will then report on the completed research, describe various components and development activities, and conclude by stating the future research plans.

#### **2 Learning Theories and Emerging Technologies**

Emerging technologies have the potential to transform the way we learn. As a result, there has been a renewed interest in learning theories that inform the design of technologymediated learning. As a framework for designing effective learning environments using advanced technologies, the constructivist theory has received considerable attention among many possible perspectives. The constructivist theory of learning was derived from the cognitive development research of psychologists Jean Piaget and Lev Vygotsky, who argued that individuals actively construct knowledge based on experience; therefore, knowledge cannot simply be handed down from one teacher to the next but rather must be constructed individually by each learner [12]. Learning Scientist David H. Jonassen states that constructivists are concerned with how learners construct knowledge, arguing that knowledge is a function of prior experiences, mental operations, and cultural constructs that help us to interpret events. He states that "constructivism views reality in the mind of the knower, that the knower constructs reality, or at least interprets it, based upon his or her appreciations" [19].

The constructivist approach to conceptualizing how we learn has several implications for our approach to developing technology-mediated environments. The idea that knowledge is a function of prior experience suggests that people develop knowledge and meaning from their experiences [1]. Therefore, the pedagogical approaches from a constructivist perspective focus on developing learner-centered, explorative, and taskoriented activities that engage the learner with interactive experiences [30]. Examples of this approach include Project-Based Learning, Game-based Learning, Situated Learning, Active Learning, and Experiential Learning. In these approaches, learners are considered the protagonists of the learning process. The learning environment is designed as a scaffolding composed of various scenarios geared to stimulate higher-level thinking skills, focusing on real-world problems [14].

#### **2.1 Game-Based VR Learning**

The constructivist theory emphasizes learner-centered experiences and activities. The literature on learning games indicates that the procedural and interactive nature of games provides an ideal mechanism for designing a constructivist curriculum based on exploration [3]. Additionally, immersive media, including VR, expands the capabilities of game engines [2], making game-based learning in VR a new research frontier. It has also been shown that immersive VR games can provide context and motivation for situated practice [6] and improve task performance [4, 25] through carefully designed game mechanics [9, 10].

Considering increasing evidence that games can motivate learners to persist in challenging tasks [16, 15, 27], engender high levels of cognitive, affective, and behavioral engagement [22], and destigmatize failure [23, 24], constructivist principles can be implemented more effectively. Specifically, virtual reality games can provide context for situated practice [25] through the careful use of game theory and the development of rigorous levels and missions [26, 27].

Design strategies for game-based VR learning include several attributes that support embodied and personalized learning. Immersive learning environments situate the learner in a sensory-rich environment and foster a sense of presence that can contextualize learning in various realistic settings, supporting situated cognition [7]. Games also facilitate simulated interactions as well as more complex and implicit cognitive engagement [29].

#### **2.2 Adaptive Learning Systems**

Constructivist learning theory also places emphasis on the individual's background and prior experiences, which prompts another set of reflections regarding the design of pedagogical approaches for technology-mediated learning [25]. According to Radianti et al. learners link new information to their prior knowledge, thereby rendering their mental representations subjective [24]. Due to this, learning environments are most effective when personalized to the individual learner's experience, aptitude, strength, and weakness [28].

Adaptive Learning Systems (ALS) are computerized learning systems that adjust learning content based on learners' profiles, performance, and human factors [18]. Although ALS have been developed and tested in the last decades, the recent advances in AI and data-driven approaches render ALS more powerful. With the application of AI, learning content can be personalized based on an individual's prior experience, performance, and proclivities. AI-powered (ALS) are increasingly able to monitor learner performance and provide personalized instruction by adapting content, tasks, and feedback for individual learning and knowledge levels.

Additionally, advances in software and hardware which have led to state-of-the-art AR and VR Head-Mounted Displays (HMD), offer the possibility to collect and harness biometric data, such as learner movement, haptic interactions, eye-gaze, and heart rate variability. Access to this data enables researchers to analyze information correlated to attention, stress, and confidence levels for improving learning and training outcomes. The ability to collect and process this type of granular data from learners has significantly improved ALS.

#### **3 Robotics Academy**

Robotics Academy is an immersive online platform designed to address the needs of students and professionals in the AEC industries. Inspired by the recent technological achievements in self-adaptive, data-driven, and autonomous systems for virtual training, the platform integrates AI, VR, and big data analytics to create a personalized experience for industrial robotics training.

In developing the Robotics Academy, we applied three main strategies. The first strategy was to research the challenges of the industry with automation and gather insights on their training needs. This was achieved by interviewing the AEC industry representatives and holding a summit to reflect on the significant industry trends and issues. This step helped us to better understand their needs for creating the appropriate curriculum, content, and the most effective delivery modality. The findings from the interviews and summit are presented in Sect. 3.1, and the resulting curriculum is discussed in Sect. 3.2.

The second strategy was to build on the principles of the Constructivist theory by developing a game-based VR learning environment that supports active and engaged learning. To achieve this goal, we have developed the learning content as a series of task and driven activities. This approach supports active engagement and exploration, as learners can conduct various tasks safely and repeat them in various contexts until they learn. This first version of the learning environment and curriculum has been developed and tested in a prototype as described in Sect. 3.3.

The third strategy was personalizing the learning process, another core concept of Constructivist theory. To achieve this goal, we are developing an ALS, which utilizes Machine Learning (ML) and Natural Language Processing (NLP) to process learners' data to build a comprehensive picture of the learner. The system learns from the activities and performance of the learner by dynamically analyzing learning preferences, skill levels, progress, biometric information, and interaction with the system. Finally, the ALS develops a recommended path through the learning content and provides feedback on the learners' activities. The development process of ALS is described in Sect. 3.4.

#### **3.1 Interviews and Industry Summit**

We began the project by conducting a series of interviews with AEC employers and employees to help us understand the impact of robotics and the training needs of the industry. These included eighteen one-hour-long, semi-structured interviews with industrial roboticists, software developers, automation engineers, system integrators, AEC educators, and product design specialists. We also interviewed analogous employees from car manufacturing and product inspection companies. Our questions for employers focused on changes in their employing pattern, jobs shifted or created, new roles and skills, industry trends, workflows, and employee training protocols.

Major conclusions drawn from the employer interviews were: (1) robotic automation is rapidly moving to take over jobs in 3D areas (dirty, dull, and dangerous); (2) new applications for robotics automation, particularly industrial robotic arms, are rapidly increasing because of their versatility and efficiency; (3) demand for skilled workers, especially those who can use new technologies and equipment, is high, but they are in short supply globally; and (4) the adoption of automation and robotics in the building industry is creating new jobs, entirely new job classes, and opportunities for entrepreneurship.

Questions for employees focused on their educational background and experience, their on-the-job training, learning preferences, training challenges, and awareness of future technologies. Key conclusions include: (1) employees do not have access to adequate training materials and they mostly learn through trial and error; (2) the use of online sources and manufacturers' manuals to run machinery, particularly robotic arms, is inefficient and time-consuming; (3) the lack of a standard or unified robotic design and manufacturing process is a major impediment to the learning process; therefore, the knowledge pool is scattered and not comprehensive; (4) employees need some level of in-person training and interaction with robots; (5) certificates and credentials leads to confidence, ability to showcase new skills, and job security.

These interviews confirmed some of our initial assumptions about the increasing rate of adoption of robotics in the industry as well as the scarcity of training materials. More importantly, it helped us develop the curriculum based on what was considered necessary for robotics training by the people actively engaged in its implementation and use.

In addition to interviews, we conducted a summit to gather representatives from the AEC fields, including the region's leading AEC firms, two of the nation's largest housing builders and developers, as well as four international experts and educators. The Summit highlighted several concerns: (1) the AEC industry faces a growing labor shortage; (2) the construction industry must dramatically improve its productivity and building performance to meet demand; (3) a transition to automated design, engineering, and construction will have severely disruptive impacts on the AEC sector with its massive potential for increased productivity leading to new jobs in robotic programming and factory-based prefabrication.

#### **3.2 Curriculum**

To design the training curriculum, we studied existing in-person and online robotics training curricula, including Kuka College, Universal Robots Academy, and ABB University. We also drew on our interview and summit findings. The resulting curriculum is composed of six modules, each containing several activities. The introductory modules focus on the fundamentals of industrial robotics, including robotic anatomy, safety, movement, and calibration.More advanced modules focus on programming, end-effector design, and integration. Each module has brief instructions and a series of tasks delivered as games with a scoring system. The modules are designed as independent and hyper-granular concepts to allow quick reorganization of the concepts by the ALS. This will allow quick reshuffling of the content based on the learner's performance.

#### **3.3 Prototype Development**

The design and deployment of the game-based VR prototype began with training students with KUKA (KR10 and KR30) and Universal Robots (UR10). These robotic arms were used to establish and test learner interactions for our game (Figs. 1, 2 and 3). The VR prototype was developed in the Unity game engine for the HTC Vive Eye head-mounted display (HMD). The steps for creating the prototype included:


**Fig. 1** Left: Scene from the VR learning prototype demonstrating lesson selection process. Right: Simulation in the Pick and Place lesson

The integration of the resulting work from these steps led to the completion of our first prototype. The next version of the prototype, which is currently underway,

**Fig. 2** Scenes from the VR learning prototype. Top: This lesson challenges the students to match the position and orientation of a "ghosted" robotic arm using both Axis-specific and World-specific motion systems. Bottom: This lesson focuses on various movements

includes programming the game mechanics, building a procedurally generated robotic training space, improving biometric data collection method from the VR HMD, and implementation of the Adaptive Learning System.

**Fig. 3** Scenes from the VR learning prototype demonstrating the learner dashboard including biometric analysis and further progress information

#### **3.4 Personalizing the Learning Environment**

One aspect of the Constructivist Learning theory is the central role of the learner's prior knowledge in constructing new knowledge. ALS has the potential to personalize the learning experience by responding dynamically to the individual learner's strengths, weaknesses, and proclivities. Thus, personalization of the learning content based on the learner's previous experience is a critical element in developing personalized learning environments.

To develop the capacity for personalizing the learning content for the individual learner, we needed to understand the individual learner in a holistic way. This required collection of several data sets to closely monitor learner progress which were: (1) Learner Profile Data including information about age, gender, ethnicity, preferred language, academic background, and skill sets in robotics, and (2) Biometric data including eye gaze and haptic interaction, (3) and Performance Data, including game interaction (task completion time, number of attempts and errors, and test and quiz scores.

Although numerous approaches for developing ALS have been described in the literature, in general they are composed of three interdependent models governed by an Adaptive Engine. These include a Learner Model, a Domain Model, and an Instructional Model [21]. Building on this established research, our project's ALS system models include:


The addition of ALS to the VR learning system promises to expand the potential of the Robotics Academy as a learning tool for industrial robotics training (Fig. 4).

**Fig. 4** Robotics Academy's Adaptive Learning Systems for Personalizing Learning in VR

# **4 Conclusion and Future Plans**

Initial testing of the Robotics Academy VR learning game prototype suggests that it is at least as effective as an in-person training course for teaching the fundamentals of working with industrial robots [23]. Furthermore, analysis of testing data shows that the platform can produce comparable, quantifiable learning outcomes to traditional settings and has the potential to serve as an effective replacement for traditional robotics training.

Further testing data analysis shows that learners who were allowed to practice specific tasks in the immersive environment could transfer VR-acquired skills to operate a physical robotics arm [23]. This reinforces our hypothesis that learning in simulated environments can support skill development for real-world applications in robotics. Additionally, the application of devised ALS promises to significantly expand the potential of the project as a learning tool. The two NSF grants have provided the opportunity to conduct foundational research, develop a plan for the Robotics Academy and create a VR learning game for training. In the following months, we plan to expand the capabilities of our prototype through the entire development and implementation of the ALS.

Since ALS requires large-scale data to function well, expanding usage and testing are critical for the success and scalability of this project. Therefore, we plan to test the Robotics Academy with 100 students, which will provide a sufficient dataset for iterative improvement and model training. The development of this learning tool will provide valuable insights into the learning process for researchers and developers studying immersive learning environments. The project will shed new light on integrating ML and NLP to develop effective ALS. Our goal is to increase opportunities for automating education and training in the AEC industries.

**Acknowledgements.** This material is based upon work supported by the National Science Foundation's Research on Emerging Technologies for Teaching and Learning program and under Award No. 2202610:*Collaborative Research: Intelligent Immersive Environments for Learning Robotics*. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.

# **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **DeepCraft: Co-Intelligent Architecture and Human and AI-Driven Craftsmanship in Design-to-Production Pipelines**

Peter Buš(B)

Institute of Future Human Habitats, Tsinghua Shenzhen International Graduate School, University Town of Shenzhen, Nanshan District, Shenzhen 518055, People's Republic of China peter\_bus@sz.tsinghua.edu.cn

**Abstract.** The working paper investigates the potential of artificial intelligence technologies (AI), namely the Generative Adversarial Imitation Learning (GAIL) implemented in a process of digital robotic fabrication prospectively to be used in craftsmanship. The method introduced is based on a preliminary demonstration provided digitally in an abstract toolpath generated by a human-driven movement in a hand gesture translated into a digital space in a real-time process. The investigation presented in this paper focuses on a preliminary computational digital framework which may serve as a base for further investigation. At this stage of the report, the framework encompasses human hand recognition creating a toolpath for a robot, which learns its principles and tries to interpret the process in a digital space. This learned toolpath resulted in a digital brain being applied again in a different shape of the human-created toolpath or gesture movement. The paper also presents the computational system of the real-time navigation of the robot based on a human gesture in a virtual space. The learned knowledge by a robot is observed in a digital environment before any physical applications.

**Keywords:** Intelligent design-to-production · Deep learning in craftsmanship · AI-driven craft · Gesture-driven manufacturing · GAIL

# **1 Introduction**

At present, the architecture, engineering, construction and operation sector (AECO) addresses a variety of challenges related to its continuous digitization, low productionefficiency in construction, sustainability, circularity, eliminating carbon emissions, and mass-customization also through the lens of Artificial Intelligence (AI) as a part of revolutionizing Industry 4.0.

Besides numerically controlled digital fabrication (NC), Building Information Modelling (BIM), design for manufacture, assembly and disassembly method of production (DfMA), and standardization of building components to create architectural scenarios (Kit-of-Parts), the AI field is starting to be prevalent to augment and extend human capabilities to deal with complex and high dimensional problems and requirements within the AECO sector too. The current advancements in these technologies place higher demands on the skills of human designers, production technologists, and construction experts to solve specific problems in mutual human-machine collaborations (Duan et al. 2017). The digitally unskilled workers, although not familiar with the latest advancements in digital fabrication, are usually highly skilled craftsmen (e.g. carpenters in timber construction production) capable of delivering unique and customized products, while preserving the traditional notion of crafts with specific human-made qualities. Thus, the craftsmen bring a qualitative value into the production pipelines, meeting the high standards as well as many qualitative criteria in hand-made processes (aesthetic qualities, detailing, smartness, and complexity of the chosen solution made by a human, hand-made quality, individually recognized authorship, specific artistic and artisanal qualities) (Pu 2020). But how to recognize those qualities in the era of technological and machine intelligence advancements?

**Fig. 1.** Diagram of the human-in-the-loop process engaging with AI technology

### **2 Artificial Intelligence in the Scope of a Craftsmanship**

This paper theoretically envisions that through human and artificial intelligence-driven processes of digital fabrication and production of artifacts where the craft skills are recognized, learned, trained, and implemented in the human-in-the-loop co-creative production workflow such as in the one-shot imitation learning (Finn et al. 2017), the technology will be capable to develop and strengthen its "wisdom" in a similar way how humans improve their skills, experience, and wisdom in time and thus, make autonomous decisions in the production process. Can a machine be capable of conceptualizing learned knowledge to yield novel artifacts through hybridized/synthesized modes of human-machine interactions utilizing Neural Networks (NN) and deep reinforced learning (DRL)?

There is a hypothesis suggesting that unique knowledge demonstrated by craftsmen can be translated into forms of demonstrations, digital models, additional data sources (such as images, videos, and sequences), and digital processes (generation of tool paths for specific digital robotic fabrication execution, unique assembly modes).

The paper further proposes that by linking human intelligence (knowledge, experiences, craft skills, capacity to make relevant decisions) and machine intelligence (responsive robotics based on multisensorial setup (Felbrich et al. 2022), XR devices and digital operations) in one coherent hybridized production loop, there will be created a novel communication and interaction platform between human and a machine via physical interventions and demonstrations leading to machine capabilities improvements to execute the production task.

At this stage, the paper introduces a computational connection between a human agent and an AI agent in a digital process to create an abstract toolpath, while theoretically envisaging a novel searching and generation method for design and production space based on a human-in-the-loop cooperative learning process (Fig. 1). In this report, the human and machine agents consider human-driven toolpath generation in a form of an intuitive gesture, while human logic, preferences, memory, cognition, and past experiences will be investigated in the next phase of the research. These aspects are tested to be transferable to a machine in a continuous training and learning process driven by a human. Consequently, the machine will constantly improve its capability based on human agents' inputs and become more autonomous in the decision-making and generation of design and production space. Such a framework aims to imitate in a re-interpreted event a human gesture, e. g. for drawing or painting intervention in a digital space.

Instead of replication and recreation of the crafting process in a numerically controlled way of digital fabrication (NC), the intention is to discover a way how the machine can express itself in a novel and augmented craft language and its formal expression in artefacts as similar as a human does, but unconventionally, beyond human imaginative solution craft space. The paper provides a preliminary conceptual digital experiment using the digital twin of the desktop robotic arm equipped with a virtual multisensorial setup. The machine learns a simple human-driven toolpath, considering gentle human movements of the hand physically provided by the demonstrator, translated into the digital space.

#### **2.1 Current Learning Experiments in the Processes of AI in the AECO and the Use of GAIL**

AI deep learning implementations in the processes of robotic digital fabrication for the potential use in the AECO sector have been tested in a variety of tasks, such as the assembly of a lap joint (Apolinarska et al. 2021) or pick and place scenarios for component assemblies (Felbrich et al. 2022). Other studies focus on co-designing strategies for autonomous construction methods (Menges and Wortmann 2022), exploring the integration of deep reinforcement learning for the intelligent behaviour of construction robots as builders.

The question of how to involve human agency in the AI-driven processes to achieve coherent results for the potential use of AI in the AECO applications on a larger scale or in human-made operations is still broadly unexplored. Imitation learning, especially Generative Adversarial Imitation Learning—GAIL (Ho and Ermon 2016) as a method to teach a robot to do a task has a solid potential to be integrated into the design-toproduction processes if we consider smaller scale in the early stage of production, such as drawing or cutting tooplath. At present, pick-and-place scenarios of simple objects using the visual demonstration and data collected from a human agent are successfully deployed (Finn et al. 2017) as a combination of the imitation and meta-learning strategies, however, the movements of the robot are still very technical and pre-programmed, although delivering the simple task successfully. The Unity and ML Agents tool to train a robot have been previously introduced by Pinochet as Smart Collaborative Agents (2020) and Hahm (2020), when the robot follows the pre-defined targets, not using imitation learning approach, and based on configurable joints or Unity articulation body.

The notion of human craftsmanship with a unique look is a very complex task to imitate or deliver, as it requires a complexity of information and data to be collected and processed. This research explores how to engage the human agent with a robot, aiming to find a method for a process to either participate together in a real-time sequence scenario or the human can act as an expert and demonstrator to teach the robot a task to execute.

# **3 Implementing the Computational Framework**

#### **3.1 Real-Time Gesture-Driven Navigation of the Robot**

There are two computational implementations described in this paper, integrating the real-time robotic navigation by a human gesture: Unity and Rhino|Grasshopper environments, which serve as an initial input for the gesture-driven toolpath generation. Both are described in detail in the following sections (Figs. 2 and 3).

The real-time navigation of a robot may have a variety of possible uses in the field of digital fabrication and manufacturing for architecture. The proposed framework can also serve as a designer's environment to be tested and explored before any manufacturing and crafting process. The data captured from a human can be stored and implemented in a custom scenario. Even though the implementation at this stage is not fully practical due to specific constraints related to noisy interference of the data exchange, the real-time interaction is engaging and can serve for further investigation in connection with the real robot. The computational models for both strategies are available from Buš (2023) and GitHub (2023).<sup>1</sup>

#### **3.2 Unity and Rhino/GH Implementation—Hand Tracking**

Both implementations encompass the User Datagram Protocol library (UDP) for the data transfer between the actual gesture and the digital environment and the robotic twin. For this implementation, the Universal Robot UR1 digital twin and a standard web camera have been used for human hand capturing.

<sup>1</sup> Buš, P. Repositories [Internet][Shenzhen], Github; 2023 [updated 2023 May 4, cited 2023 May 5]. Available from https://github.com/peterbus?tab=repositories.

#### 372 P. Buš

The computational approach of the hand recognition and the data exchange platform in Unity implement the Unity UDP receiver script, provided by the CV Zone platform as an open data resource (Murtaza 2022). The Unity and Rhino|Grasshopper environments were customized and adapted for the robotic movement. Both strategies utilize the CVZone Hand Tracking module with the Hand detector implemented in Python to recognize the hand of a human (Murtaza 2022). Hand recognition contains 21 points which are interconnected with lines, representing the virtual skeleton of the human hand.

**Fig. 2.** Hand tracking implementation in Unity connected to the UR robot. The robot follows the finger in real-time process while properly rotating based on configurable joints

#### **3.3 Unity Basic Setup**

The 21 recognized points are transferred utilizing the UDP data protocol into the Unity environment using the local host. The data are constantly received by the receiver and the points are embedded as Game Objects creating the foundation for the skeleton.

The specific point can be selected as a spawner of the checkpoints for the robotic toolpath. Based on human movement, the hand spawns the targets for the robot, specifically rotated according to the hand movement. Custom C# scripts were written to link the hand with the digital model of the UR robot, which is based on configurable joints for each of its axes. As such, it was possible to create a Target for the robot, which follows it. In that way, the robot is navigated by the hand point on the selected finger in real time, considering the physics engine in Unity and rotating and moving based on the customized configurable joints.

#### **3.4 Rhino|Grasshopper Setup**

Similarly, the recognized hand points were transferred via the UDP protocol into the Rhino|Grasshopper environment and points were reconnected. This was done as an independent platform. For the UDP communication transfer, the GHowl addon has been used (Alomar et al. 2011), considering the position of the points as well as the distance information between the hand and the web camera.

Using this information, it was possible to implement the third dimension to navigate a virtual end effector of the robot in all three dimensions. For the robotic real-time simulation of the moving robot, the Robot addon was utilized. The GH definition can serve as a starting point and a test bed for further implementations and testing purposes. The working version of the GH definition is available2 (Buš 2023; GitHub 2023).

**Fig. 3.** Testing the robot and gesture movements in real-time. The hand spawns the targets for the robotic toolpath, according to the direction of the hand movements. The framework was implemented in Unity and Rhino Grasshopper utilizing UDP, Ghowl and Robots addons

#### **3.5 GAIL and Behavioral Cloning Test in Unity and Observation**

The Unity environment was further tested to teach the robot to recognize the human gesture and interpret it afterward it was captured. Several custom scripts were developed to do so as well as the standard toolpath following a system based on numerically controlled positions. These were captured from the spawned checkpoints from the human gesture to create a linear toolpath. However, the implementation of the deep learning method, in this case, imitation learning utilizing the ML-Agents tool in Unity with the GAIL learning method combined with behavioral cloning was tested and observed (Juliani et al. 2018).

The GAIL (Ho and Ermon 2016) considers the policy from the expert demonstration to perform a task based on 'how to act by directly learning a policy' from the data provided. The ML-Agents tool contains an imitation learning approach utilizing the GAIL and the behavioral cloning method, which aims to capture the pre-defined process of a demonstration of how the robot should perform the task according to the expert demonstration. It follows the sequence of targets in a toolpath, previously generated by human in real time. In this experiment, the data captured from the gesture served as

<sup>2</sup> Buš, P. Hand-Tracking-to-navigate-a-robot-Rhino-Grasshopper-framework.[Internet][Shenzhen], Github; 2023 [updated 2023 May 4, cited 2023 May 5]. Available from https://github.com/pet erbus/Hand-Tracking-to-navigate-a-robot--Rhino-Grasshopper-framework.

an input for the demonstration recording, containing the transform position information (transform position, rotation, scale) of spawned targets from the gesture.

The positions were translated into the toolpath, and a virtual ML agent run through them several times (see Fig. 4). The agent can serve later as an input for the robotic end effector target, mentioned above. The heuristics training simulation contains the digital demonstration, captured as a demo for the GAIL and behavioral cloning training.

The training algorithm included the default one based on the Proximal policy optimization hyperparameters (PPO) for the ML agent, tested with different setups for the GAIL strength or behavioral cloning. The virtual agent looks randomly for the checkpoint positions in space and learns from them how to interact with them in each of the episodes. The task for the agent was to recognize starting position and end position as well as checkpoints to perform the toolpath in the right order and direction. In addition, each of the iterations slightly randomly moves the positions of the path checkpoints to encourage the agent to learn from these novel positions. This might serve for the potential future gestures that will be each time different.

**Fig. 4.** The virtual end effector follows the agent, while running through the checkpoints on the toolpath

The learning process contained 3–5 million iterations (steps) with a positive or negative reward structure for the agent, each time it collides with the correct or wrong checkpoint. The process generated the virtual brain for future testing scenarios. As it was qualitatively observed from the preliminary tests in the learned positions of the agent in the final inference training, the results with the current setups do not precisely imitate the original demonstration, although the agent reaches the targets in the right directions and with the right orientation in a sequential way. The quantitative results are provided in the following scalars, captured from the Tensorboard platform (TensorFlow 2023), showing relevant reward processes and GAIL policies. While the cumulative reward is decreasing (in case the model was trained without extrinsic rewards), the GAIL Loss and Pretraining Loss showed some models adopted well according to the demonstration, as the curve slightly decreases in time, assuming the agent learns the policy. The GAIL reward increased after a certain number of iterations and the agent obtained relevant rewards while learning the policy. There was a big decrease observed at the beginning of the training process—it depends on a variety of combinations of hyperparameters set in the configuration file. The training delivers a variety of brains with less or acceptable training results. During the training, each scenario had a moment of decreasing the reward value, which later became more stabilized. In addition, the agent continuously improves the imitation of the demonstration during the training duration (Fig. 5).

**Fig. 5.** Preliminary GAIL training results of the agent in several running scenarios. Its scalars show the reward and episode length increase, although there is a big drop in the first part of the training process. The Policy loss is not convincing at the preliminary stage for some cases-it should rather decrease. Additional set of observations related to GAIL policies

#### **4 Discussion and Further Potential**

Although the preliminary training of the agent to follow the toolpath is not satisfactory enough, the Unity-based computational framework might serve as a base for further testing and observations. So far only one type of algorithm has been tested, namely the Proximal Policy Optimization (PPO), which uses a neural network to approximate the ideal function that maps an agent's observations to the best action an agent can take in a given state (Juliani et al. 2018).

The other algorithms and different hyperparameters can be tested and evaluated according to the specific needs of the designer, such as testing different strengths of GAIL or behavioral cloning and their combinations. The potential of the human hand, movements, and gesture recognition lies in the prospective implementation in the making and crafting processes when the hand and movements of the craftsman can be captured and recognized to inform the learning policy in a form of an expert demonstration.

At this stage of the investigation, the robot movement is not straightforward, as it contains a certain noise, which prevents the robot from moving smoothly as in the demonstration. This can be addressed by a higher number of training episodes during the default training (this also requires longer training time), and a higher number of steps in the demonstration data. The robot itself can be set up through an updated articulation body tool in Unity, benefiting from Unity physics instead of the current setup of configurable joints. This will improve the motion of the robot. Even though the gesture is not precisely cloned by AI, the resulting digital process partially follows the human inputs because of a pre-trained process. In the next phase of the research, such an approach can be used to train the AI in the assembly of spatial scenarios based on components deployed as kit-of-parts to create a spatial configuration. In the context of the AECO, this might contribute to the space created in unconstrained construction site conditions, while considering the human aspect in creating a unique space.

The future research will concentrate also on the demonstrations provided by the craftsmen, utilizing more advanced recognition-based sensorial setups, such as motion capture methods and tactile sensors to obtain more precise data. These will be integrated with the Unity framework. From the preliminary results, the author observed that the GAIL combined with behavioral cloning (the strength ratio implemented was 0.5 for both reward signals-GAIL and behavioral cloning) has the potential in digital fabrication and production processes, however, more tasks and more robust processes must be tested first, such as the creation of an assembly based on kit-of-parts system.

# **5 Conclusion**

This working paper introduced preliminary computational frameworks for further testing and observations, potentially to be deployed in handcrafting or assembly processes, utilizing digital tools, such as collaborative robots.

The environments such as Unity and Rhinoceros can serve as platforms to integrate more gentile operations in making, based on handcrafting, followed and learned by AI. Even though this hypothesis has not been fully proved yet as the computational models need further development and testing, it may be argued that hands-on operations followed by AI-driven technologies will shift the way how the crafting processes can be executed in the future and will bring novel understanding where the human agent is still an expert and an important production agency in the human-in-the-loop processes.

Preliminary observations of the virtual hand proved satisfactory real-time navigation of the robot (without a specific sensorial framework), however, further testing with the physical robot is necessary to fully prove the concept.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Making Matter: Small-Scale Biomorphogenic Prototype Based on Ulva-Algae-Biopolymer**

Haoyi Chen(B) and Claudia Pasquero

Synthetic Landscape Lab, IOUD, University of Innsbruck, Technikerstraße 21, 6020 Innsbruck, Austria

Haoyi.Chen@student.uibk.ac.at, Claudia.Pasquero@uibk.ac.at

**Abstract.** Recent developments in digital architecture have placed a renewed focus on sustainable architectural materials and the circularity of material systems. Algae has emerged as a promising material for mitigating the effects of climate change due to its ability to absorb large amounts of carbon dioxide. However, the disposal of algal biomass can lead to significant CO2 emissions and air pollution. The upcycling of algae into composite materials can promote circular economies by reducing the demand for petroleum-based products. In this context, this research explores the potential of Ulva algae in creating 3D-printed architectural prototypes based on bio-algorithm. An experimental analysis of the material properties of algae-based plastic is conducted and compared to similar reference products. This study argues for the importance of designing and fabricating these materials at the required scale while leveraging bio-thinking principles to create closed-loop systems and maximize the potential of natural resources.

**Keywords:** Prototype · Making matter · Algae · 3d print · Bio-algorithm

# **1 Introduction**

Due to the fast-growing urbanization along with increased climate change impact, the demand for sustainable building materials also continues to grow. In recent years we have witnessed a new emphasis in digital architecture on novel modes of production as well as material computation and circularity. As a result, fabrication techniques and material systems originally not part of the architectural realm can now become an integral part of the design workflow (Gazit 2016) as well as design philosophy (Pasquero and Zaroukas 2016). Accordingly to the essay by Michael Weinstock in the book Systemic Architecture by Poletto and Pasquero (2012), the convergence of architectural forms and practices worldwide can be attributed to various factors, including the interaction of computational systems, the transmission of information through the internet, and the rise of a global rapid transit network. As architects and urban designers navigate the rapidly emerging and complex material world, they find themselves challenged by the intellectual insights and knowledge they must produce, evaluate, and distribute (Pasquero and Zaroukas 2016). In the face of these difficulties, they seek to find a mediating force that can bridge their understanding of the factors present in the interplay between the human and non-human.

Pasquero and Zaroukas (2016) argue that every prototype, in its function as a "weird media," refracts its inputs by materializing new entities. As a result, the prototype extends the human sensorium and reconstitutes an augmented and transformed agent that is capable of experiencing more than a human subject alone. This is the potential of prototypes functioning as weird media.

In response to these observations, ecologicStudio has developed bio-architectural prototypes that demonstrate this potential. The Urban Algae Folly, for example, is an interactive pavilion that integrates living micro-algal cultures. The effective translucency, color, reflectivity, sound, and productivity of the Folly are the result of the symbiotic relationship between climate, microalgae, humans, and digital control systems (Pasquero and Zaroukas 2016). The H.O.T.U.S project explores the relationship between urban renewable energy and agriculture through a new gardening prototype (Pasquero and Zaroukas 2016), while the Tree.ONE project, exhibited in Hyundai Motor Studio in Busan (EcologicStudio 2022), showcases a "living tree" prototype designed by artificial intelligence and bio-digitally grown with algae-based biopolymers that capture carbon and promote a pollution-free and carbon-neutral environment (Fig. 1).

**Fig. 1.** Urban algae folly, H.O.R.T.U.S, Tree.One (Provided by ecologicStudio)

These prototypes are not simply machines, but rather construct an interactive ecology of their own. They bring together human and non-human agents, both organic and inorganic (Barad 2012). The production of knowledge in this context is saturated with human metaphors and images, yet it also bears traces of the inhuman. In this sense, the world becomes an intra-active ecology, and prototypes are seen as apparatuses that construct the categories of human and non-human (Pasquero and Zaroukas 2016).

Algae is acknowledged as one of the most rapidly growing plants on the planet and is seen as a promising material in the efforts to mitigate the effects of climate change. This is due to its ability to absorb a minimum of 10 times the amount of carbon dioxide compared to other plants (Batista et al. 2015). However, due to climate change, for several decades, the coastal seas of a growing number of countries covering many areas of the world have been affected by blooms of algae (Lassus et al. 2015). The Ulva algae booming pollution often affects major coastal cities like Qingdao, China. To remove tonnes of it from beaches in Qingdao, the city's government dispatched thousands of boats and bulldozers in 2021 (Zhang et al. 2019) and transfer the rural areas to dispose (Madejón et al. 2022). The disposal of this material in landfills or by incineration and burial can result in significant CO2 emissions and air pollution (Castaldi and Melis 2004). This is due to the production of methane, a potent greenhouse gas, from the organic matter in the algae during decomposition in landfills (Chisti 2007), as well as the release of CO2 and other pollutants from incineration and burial (Sánchez-Martín et al. 2018). To mitigate these environmental impacts, converting the material into composite materials has been suggested, which can reduce the demand for petroleum-based products and promote sustainability (Wang and Wang 2017) (Fig. 2).

**Fig. 2.** An excavator removed algae at a coast in Qingdao, China (Image by Dan Hei)

Given the pressing environmental challenges we face, Steven et al. (2020) argues that biodegradable and bio-based plastics offer a promising solution for promoting sustainable growth within the plastic industry and serve as a feasible alternative to petrochemical plastics soon. From an ecological standpoint, these materials exhibit biodegradability and have the capability to sequester carbon dioxide, thereby reducing the carbon footprint associated with a product or structure over its lifecycle (Steven et al. 2020).

Consequently, the present research endeavors to investigate the possibilities of upcycling the material system for architectural prototypes utilizing the recycled Ulva algae. The design is to promote ecological balance and to consider the process of "making matters" as an integral aspect of the design process. From Ulva algae leaf to algae powder, from algae powder to algae-based plastic, and the 3d printable algae filament. We investigate the material properties of algae-based biopolymer through a quantitative experimental analysis and comparison with comparable reference products. Our goal is to explore the relationship between material making, design methodology, and biology by integrating natural organic forms with materials in architectural prototypes. This approach will reveal new design possibilities at various scales, from micro to macro. To achieve this, it is essential to tailor the design and fabrication process to the specific scale required. This is because a "one size fits all" approach is not practical in digital fabrication (Yeang 2008). To evaluate the performance and materiality of algae-based plastic as a building material, we use digital morphology to create 3D-printed prototypes based on human body geometry, serving as the data origin for micro-scale space (Fig. 3).

**Fig. 3.** 3d printed coral morphological prototype based on algal biopolymer (Author)

This paper argues that the integration of bio-thinking principles in design has significant potential to create closed-loop systems within a circular system, leveraging natural resources such as Ulva algae. The goal is to create an architectural prototype where the storage and reduction of carbon emissions is achieved through a balance of resources taken and returned to the environment.

# **2 Making Matter: Algae Based Biopolymers**

Alva lactuca (sea lettuce), a green edible alga in the Ulvaceae family (Bates 2020), is utilized in food, agriculture, and medicine industries (Groenendijk et al. 2016). However, it also contributes to harmful algal blooms (HABs) in coastal cities such as Qingdao, China. HABs can impact aquatic life, causing difficulty in finding food and population displacement. When transported to rural areas for disposal, such as in landfills or through incineration and burial (Madejón et al. 2022), it results in additional carbon dioxide emissions and air pollution.

This algae plastic is made of carbon that has been drawn from the carbon reservoir of the atmosphere and put into the stock of carbon of our built environment. -Charlotte McCurdy (Hahn 2019).

The use of algae as a platform for bio-polymers production offers direct carbon capture by removing CO2 from the atmosphere through the photosynthesis process and conversion to various forms in central metabolism (Satyanarayana et al. 2011). Our study focuses on the development of sustainable materials based on algal biomass that capture and consume CO2 and nutrients. The adoption of this methodology also entails the utilization of otherwise incinerated algae through recycling, resulting in a decrease in carbon emissions, as demonstrated by the project of Tree.One and Otrivin Airlab from the EcologicStudio (2022).

#### **2.1 Material Making Experiments**

According to Rahman and Miller (2017), four methods exist for converting algae into bioplastics: direct conversion, genetic engineering, blending with petroleum, and blending with bioplastics. The use of biological feedstocks can result in plastic with similar performance characteristics as those produced from fossil fuels. However, the observation from us shows that the Ulva Lactuca algae has a potential size of 60 cm, but typically grows to less than 30 cm, making it unsuitable for material composite testing.

Yaradoddi et al. (2016) conducted research on developing biodegradable plastic from waste fruit. In this study, dry Ulva algae was ground into powder and added to the traditional corn starch-based bioplastic production process. The study used a constant set of dosage parameters of potato starch, water, and lemon vinegar for each set of testing samples. The experiment included two control groups, with varying doses of algae powder and the addition of glycerin.

The material properties of the algae-based bioplastic were observed, and the results showed that the addition of Ulva algae powder and glycerin significantly impacted the material's properties. The best flatness and malleability of the material were observed with the addition of algae powder in the middle range (Fig. 4). The first control group lacked glycerin and exhibited high brittleness, while the second control group, with glycerin, had a rubber-like softness (Fig. 5). Overall, the addition of Ulva algae and glycerin improved the material's flexibility and quality.

#### **2.2 Algae-Filament Making Process**

We harvest carbon dioxide, purify polluted urban air, and 3D print plastic free biodegradable products. Essentially, we convert air pollution into products that help protect our breathing. -Claudia Pasquero and Poletto (2022)

By integrating algorithmic design and 3D printing with algae filaments, the project of Otrivin Air Lab proposes a bio-digital fabrication process using bioplastics and biorubbers to 3D print algae filaments to create the NetiPot as a new line of products that update the historical evolution of nasal cleansing and respiratory wellness. The ecologicStudio designers have developed a new nasal cleaner by rotating the surface with varying angles

**Fig. 5.** Algae plastic making experiment (author)

and rotations. This 3D printed algae-based bottles offers a distinctive grip and rejuvenates the aesthetic view of such cleaners. The material's color deepens with increasing layer thickness, providing excellent waterproofing and strength, even able to bear the weight of an adult. Furthermore, the 3D printed stool made from algae filament shows high support strength in large-scale testing.

The integration of Algae-filaments from Algae-plastic with the cutting-edge 3D printing technology constitutes a pioneering methodology for assessing the physical properties and behavior of algae-based plastic through the lens of data-driven digital morphological analysis. This methodology offers a unique opportunity to experiment with and evaluate the performance of algae-based materials in a controlled, computational environment (Fig. 6).

**Fig. 6.** Otrivin Airlab, photo by @NAARO (Provided by ecologicStudio 2022)

In order to create a wider variety and range of products based on discarded Ulva algae, we have decided to customize our own 3D printing filament. This will enable us to test a greater number of material properties based on varying amounts of different types of seaweed additives, and to more flexibly explore the potential for creating a greater diversity of architectural forms. In our experiment on the production of algae-based plastic, we employed various industrial tools such as the FLD-25A filament extruder, an electric heating blasting drying oven, and also introduced dried Ulva algae powder mixed with Biopolymer 4032D (NatureWorks) to generate a 3D printable algae filament. The filament-making process was compartmentalized into three distinct stages, each consisting of 14 procedural steps: material preparation, material blending in accordance with specified ratios, and filament extrusion. Our current efforts in production entailed evaluating the influence of varying three different proportions of algae in biopolymer pellets on filament properties under constant extrusion conditions, including temperature and extrusion speed, as illustrated in the flowchart presented in Figs. 7 and 8.

Comparison of results (Fig. 9) showed that the ratios of algae additions impact the color and visibility of algae impurities. As the amount of algae increases, the material becomes progressively darker and takes on a brownish-green colour. Ensuring that the material base temperature is appropriately set to the standard PLA extruding temperature is crucial in preventing the plastic from being scorched as a result of excessive combustion. Meanwhile extruding the filament too quickly or too slowly can both lead to deformation of the filament and uneven distribution of the seaweed powder. The algae filament was found to be relatively brittle compared to commercially available standard PLA filament. According to Bulota and Budtova (2015), the addition of algae flakes decreased the tensile strength of the material in all cases, from 65 MPa for neat PLA to around 30 MPa at 40 wt% of filler, regardless of the type of algae or particle size. Hence, it is imperative to exercise caution in determining the appropriate quantity of seaweed to be incorporated. Next, we increased the ratio to 3%, but the filament did not form and had an irregular shape (Fig. 9). However, by adding 0.1% plasticizer to the biopolymer, the sample (Fig. 9, far right) became more formable, smoother, and less brittle compared to the previous three.

The experiment results were analyzed, leading to several modifications to the material. The drying time of Ulva algae was reduced to 14 days, the melting temperature of the extrusion was lowered to 190 °C, the toughening agent was maintained at a ratio

**Fig. 7.** Algal filament making preparation (Author)

**Fig. 8.** Algal filament making process (Author)

of 0.1:100, and the algae/biopolymer ratio was increased to 10%. The sample produced (Fig. 10) displayed a green color, similar to Ulva algae, with a smooth surface and a slight matte texture. Additionally, the material's toughness improved compared to previous results.

**Fig. 9.** Material properties of algal filaments with 0.1, 0.3, 0.6 and 1 algae ratio (author)

**Fig. 10.** Updated Algal filament (10%) and printing detail of this material (author)

# **3 Making Matter: 3d Printable Bio-Morphological Prototype**

Nature is a profuse and ample reservoir of inspiration for innovation, comprising both visual and functional elements. Bio-digital architecture synthesizes biological and digital techniques, seamlessly fusing natural intelligence with computational technology (Estévez and Navarr 2017). Jan Kaplický, who believed that architecture should be inspired by nature and reflect its principles of organic growth, argued that the use of bioalgorithms in architecture can result in structures that are not only functional and efficient but also aesthetically pleasing (Kaplický 2005). Furthermore, the use of bio-algorithms represents a step towards a more dynamic and adaptable form of architecture that can respond to changing environmental and cultural conditions (Deleuze 1990). Additionally, incorporating bio-morphology into architectural prototypes offers the advantage of optimized material use, as the forms and structures in nature are known to be highly efficient and optimized for their intended purpose (Dey and Dutta 2015). The integration of bio-algorithms in architecture does not involve the use of biology as a mere metaphor or the emulation of biological structures. Rather, it involves the incorporation of the algorithmic structure and the logic of spatial interaction, negotiation, and self-organization inherent in these algorithms. The appropriated behaviors from bio-algorithms range from pragmatic rules regarding structural and programmatic adjacencies to abstract behaviors that produce micro to macro scale architectural effects.

Coral reefs provide microenvironments for sea creatures due to their natural colony behavior on sea rocks. This ecosystem can range from small solitary organisms to large colonial islands. The H.O.R.T.U.S. XL Astaxanthin.g project by ecologicStudio is a large-scale, high-resolution 3D printed bio-sculpture that supports both human and nonhuman life. The substratum morphogenesis is simulated through a digital algorithm that mimics coral colonies (EcologicStudio 2019) (Fig. 11).

**Fig. 11.** H.O.R.T.U.S. XL Astaxanthin.g at Centre Pompidou in Paris, France, The algorithmic design technique inspired by the morphogenesis of coral colonies (Provided by EcologicStudio 2019)

The digital algorithm of coral colonies, referred to as recursive growth, has the ability to generate architectural prototypes at various scales to evaluate the behavior and performance of Ulva algae filaments in different functional uses and spatial data. This exploration begins with a microscale spatial data analysis of the human body. The human body geometry is a crucial aspect in bio-digital architecture, serving as a singular source of spatial information and geometry. The digitalization of the human body creates a spatial network such as points data or mesh data linking bio-morphology and biomaterial. This has been emphasized in studies of bio-digital architecture (Kontovourkis et al. 2015), which highlight the potential of using the human body geometry in architectural prototyping and design. The use of 3D printing technology, such as FDM, allows for the quick and efficient fabrication of physical prototypes, enabling designers and engineers to test and refine their designs in a fast and cost-effective manner.

# **3.1 Design Prototype Generation**

In the design of coral morphogenesis, the recursive algorithm is utilized to simulate geometric growth with constant diffusive and recursive features. Recursion, which refers to the recurring phenomenon governed by natural forces, is applied in growth from the molecular level to the cosmos (Dandu 2019). To understand this process, we generate diagrams of the generating process. We test the algorithm's spatial data and parameters by adjusting the pattern of digital morphogenesis on a single polygon sphere (Radius = 15 cm) through Surface Subdivision (smooth degree and resolution), Curvature Gradient (Magnitude of morphological change diffusion), Curvature Gradient Scale (Density of morphological change diffusion), Iteration (number of model recursions), and Voxel Size (output resolution). For comparison, we set the Volume Velocity, which is based on Perlin noise, at a constant value to determine the direction and degree of twisting and growth (Fig. 12).

**Fig. 12.** Morphological generation catalogue based on algorithm of coral growth (author)

The digital simulation of morphology and comparison with reference groups allow us to determine the impact of parameters on morphological growth. Increasing the sphere's subdivision (from 20 to 60) results in increased density and connectivity of branches. Comparing reference groups (rows 2, 3, and 4) shows that reducing the Curvature Gradient enhances diffusion area and resolution, while increasing Gradient scale increases model variation strength. At iteration 35, it becomes impossible to maintain the sphere's basic state.

#### **3.2 Generating Prototype on Human Body**

The digital representation of human geometry can be conceptualized as a variable and dynamic polygon surface that is capable of being deciphered. The human body geometry is divided into Catmull-Clark subdivision, and intersections between these planes are assigned with spatial coordinates. Through the implementation of data filtering, specific regions of the human body can be selected and utilized to generate the morphogenesis of coral colonies.

The analysis of the diagram and prototype demonstrates that the human body geometry aligns with the sphere-based simulation logic, but parameters must be adjusted to account for feasibility of fabrication and body fit. The curvature scale should be moderate to prevent over-curvature, and a moderate voxel size and curvature gradient must be set to ensure 3D printing feasibility and prevent excessively thin or dense bifurcations (Figs. 13 and 14).

**Fig. 13.** Morphological generation catalogue based on algorithm of coral growth on digital human body geometry (author)

#### **3.3 Fabrication (3D printing) Process Based on Ulva Algal Filament**

In the final section, the performance of algae-based materials in small-scale human body applications through FDM 3D printing technology (Anycubic Kobra Max) is investigated. The algae-based filament utilized is a synthetic bioplastic with a biological base, thus the printing temperature was set at 195 °C to ensure proper melting of the PLA and maintain the properties of the dried seaweed powder (Wang et al. 2009). Other printing parameters, including the printing speed of 50 and the printing bed temperature of 70 °C, were based on the characteristics of PLA. Multiple print tests were conducted to determine the most efficient and optimal parameters for the fabrication. The final parameters selected, based on surface integrity and efficiency, include the layer height of 0.4mm, the wall layer of 1, the wall thickness of 0.4, the infill density of 7% and the 10% zigzag support (Fig. 15).

**Fig. 14.** Decoded human body mesh geometry integrated with design prototype(author)

**Fig. 15.** Fabrication workflow by FDM 3d print technology (author)

In evaluating 3D printed prototypes integrated with human body geometry, the fit to the body surface and comfort achieved through the smoothness of the algae filament are satisfactory. The prototype weight of 340g does not pose a burden. Potential improvements include reducing layer height for improved surface accuracy and adjusting print angle to decrease support density for material efficiency.

The integration of Algal bioplastics with 3D printing has the potential to expand its use in various applications, including wearable devices as mobile energy sources utilizing human carriers as architecture. The precision of 3D printing allows for the creation of customized devices that fit the individual's body shape and needs (Muldoon et al. 2022). Further investigations through extensive experimentation is essential to improve the development of the technology in question. In the realm of digital fabrication, it is imperative to understand that a homogeneous approach is not tenable. As such, designers should adopt a process that involves iterative design and trial of prototypes at various scales, with a view to effectively exploring the spatial attributes of the technology (Fig. 16).

**Fig. 16.** Printing details of design prototype (author)

# **4 Conclusion**

In conclusion, this paper posits that incorporating bio-thinking principles into the design process has the potential to promote ecological balance and emphasize the integral role of material making in the design process. Through the utilization of natural resources such as Ulva algae to create biopolymers, the integration of bio-thinking principles into the design process can lead to the development of bio-morphological prototypes that mediate between the human and nonhuman. This, in turn, provides a speculative glimpse into the future of architectural material systems and the potential for closedloop systems within a circular economies. The resulting architectural prototype not only demonstrates the storage and reduction of carbon emissions, but also highlights the importance of balancing the resources taken from and returned to the environment.

# **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Animate Concrete: Materialization of Concrete Element Kinetic Assemblies**

Samim Mehdizadeh(B) and Oliver Tessmann

Digital Design Unit (DDU), Technical University of Darmstadt, Darmstadt, Germany mehdizadeh@tu-darmstadt.de

**Abstract.** Animate Concrete informs building elements for motion and future reuse. This paper gives technical insight into strategies to reconfigure building systems with lightweight and movable concrete elements. Animate Concrete asks, what if architecture becomes an ever-changing system built with lightweight but heavy-looking elements that can move, assemble and disassemble through a gentle human touch? This vision allows for a versatile space, adaptation, and reconfigurability. Animate Concrete furthermore seeks to provide novel strategies to minimize material consumption for building elements by rotoforming thereby significantly reducing the weight of robotically precast concrete elements.

**Keywords:** Sequential dynamic casting · RotoForm · Design for reuse · Digital concrete · Programable material · Center of mass · Motion capture · Robotic fabrication

# **1 Introduction**

#### **1.1 Reconfiguration and adaptability**

Concrete is responsible for an enormous amount of CO2 emissions on our planet. The construction industry is responsible for a major part of global material consumption, energy use and waste production. The harmful environmental impact of the building sector is caused by the linear chain process from material extraction to construction and demolishing without any preconception on the future reuse (Fig. 1).

The urge to act against climate change expands the existing design strategies in the field of reuse and designing with available material in the context of computational design and robotic fabrication. However, our future models for design and building with existing materials must be changed to offer the ability for reuse. What we build today should be reusable in the future. Animate Concrete is motivated by the idea of design for reuse and investigates the assembly models for future design with concrete elements.

Prefab concrete was the solution to the post WWII housing crisis in eastern Europe. The illustrations of flying concrete slabs hanging on the cranes were celebrated as futuristic, high-tech solution for the housing problem (Alonso and Palmarola 2019). It had solved the urging problem of its own time. However, the modularity of those structures could not be used for future disassembly. Once obsolete, these vast structures are not

**Fig. 1.** The installation/performance "Animate Concrete" with five 1:1 scale robotically fabricated hollow concrete element in motion and interaction with humans. Collaboration of Samim Mehdizadeh, Oliver Tessmann, (DDU, TU-Da) with Ashkan Afsharian, Johanna Kasparowitsch (Tanzpol Berlin), (*Animate Concrete DDU with TANZPOL*, 2022)

disassembled but demolished and replaced by new buildings consuming new materials. This problem motivated the researchers to propose and build a system that offers reversible connections for an ever-changing architecture. The elements of Animate Concrete appear solid but are not heavy, they are load-bearing but not static parts, designed and built to be elegantly moved and reconfigured in space.

Recent research addresses the systematic investigations to minimize the material consumption for concrete structures through novel computational design methods. For instance Block research Group at ETH Zürich addresses the Fabrication Techniques for Thin concrete Shells (López et al. 2014; Rippmann et al. 2018). Digital technologies have made the fabrication of the building part fabrication more efficient in terms of material consumption. Integrating robotic fabrication in the concrete construction field promises an automated pipeline for producing concrete elements with fewer labor costs, less material consumption, and faster in the interest of the construction sector. Building lightweight with concrete is an essential first step to minimize material consumption. However, current real estate value cycles require solutions for reuse scenarios to be considered in the design phase already.

This paper presents a design for the fabrication process for reusing concrete parts in a proposal for ever-changing architecture. The advantage of lightweight concrete elements can be leveraged by efficient material distribution inside the concrete elements to allow a continuous reassembly, reconfiguration, and permanent change by its users.

The research "Animate Concrete" systematically seeks a method for generating building elements with diverse motion behavior for a self-guided assembly. Like characters they move, wobble and rotate—behavior that becomes possible by partly filling their hollow cavities resulting from the rotoforming fabrication process. The characters are differentiated not only through their appearance, shape, size, and geometry but also through their embedded and invisible attributes, called behavior in the motion. Eclectic motion characteristics become possible through the embedded mathematical and physical properties of counter-intuitive locations of the center of mass and the resulting moment of inertia (Fig. 2).

**Fig. 2.** A various scenario for re-assembly of concrete elements through their assembly sequence and motion behavior.

# **2 Research Background**

#### **2.1 Discrete design and Self-assembly**

Discrete design entails an opportunity for assembly and re-assembly thinking (Retsin 2016).The work of Skylar Tidbits at MIT shows opportunities for self-guided assembly by manipulating discretized elements' physical and geometrical properties (Tibbits 2012). Tibbits proposes a future model of assembly, where the complex fabrication and geometrical logic adds the function (self-assembly) to the discretized elements rather than adding any additional physical parts. The notion of "designing for self-assembly" demonstrates the ability to embed assembly information in the building parts by defining their physical properties in the design-fabrication framework (Lendlein and Trask 2018).

#### **2.2 Architectural Character; Behavioral Diversity**

The research project Ruben Structures, Manja Van de Worp discusses the structural performance of building systems (so-called Rubens Structures) in which the same elements perform differently based on their position and orientation in the overall structure (van de Worp 2019). The example of a concrete chair examines a load-bearing element with diverse structural performance in the different positions of the element. The radical shift from singular structural capacity to multiple performance scenarios enables the reuse of the elements in the different load cases. What might, until now, be regarded as an over-dimensioning of structures could mean that an Architectural Character might act as a beam or a column according to its position, orientation and load transfer situation and its required structural function.

#### **2.3 Design with Forces/Weight**

Designing with weight is not particularly new to architects and engineers. A prominent example of that is the work of Gaudi with the hanging model as a physical and physicsdriven design approach. In the digital era, calculation and simulation have become easily accessible for architects. The emerging digital tools enable architects and designers to implement complex mathematics and physical models as a design driver. The contributions of Daniel Piker's plug-in Kangaroo (physics simulation) made the force an accessible design tool more than ever for educators and research institutes. Brandon Clifford's "The McNelly Megalith" project demonstrates a novel understanding of the archeological secrets of his research on the geometrical and how the giant monolithic stones can be moved by humans by adjusting the center of Mass (CoM) and the geometry (Clifford 2016). The weight manipulation of concrete elements enables the movement of colossal monolithic concrete elements blocks with a small force, as shown in the Walking Assembly project (Swingle et al. 2020).

The mathematical relationship between the weight distribution inside objects and their motion behavior have been investigated by researchers from Disney Research Zurich and the ETH Zurich (Bächer et al. 2014). Bächer et al. developed a generative optimization algorithm to maximize the moment of inertia inside hollow cartoon characters and to calculate a material distribution that allows a spinning object to be in a dynamic equilibrium while revolving around its unstable tip. Furthermore Musialski et al. offer a computational optimization method to vary the offset of a surface with the object of reaching the equilibrium with shifting the CoM (Musialski et al. 2015).

#### **2.4 Digital Concrete and Robotic Fabrication**

The extensive research and advances in computational design and robotic fabrication with concrete illustrate the opportunities and necessity of changing the design and construction methods (Buswell et al. 2020). The enormous potential of digital fabrication with concrete is not limited to 3D printing; instead, other digital fabrication methods have been developed and discussed in the last years by a vast range of researchers (Wagner et al. 2016). Digital fabrication techniques are strongly associated with material placement techniques (e.g. 3D, printing with concrete) but also revisiting and remodeling existing materialization methods, such as slip casting with robotic fabrication technologies (Lloret-Fritschi et al. 2020). The research project Smart Dynamic Casting illustrates a method for using the robotic trajectory as a generative tool for forming concrete.

The research track robotic sequential casting (RotoForm) by (Mehdizadeh et al. 2022) utilizes the robotic trajectory and the material flow behavior to shape the cavities inside the hollow concrete elements. RotoForm is an automated fabrication method for materializing hollow concrete elements (Tessmann and Mehdizadeh 2019). A minimum amount of material is poured inside the formwork, enough to cover the surface with a thin concrete shell. A robot then gently rotates the formwork so that the material flows along its surface. Repeating this action in sequences with differentiated robot trajectories allows for uneven wall thickness and material distribution inside the formwork. It enables manipulation of the center of mass inside the hollow concrete elements.

# **3 Method**

There are three essential scopes to develop to offer an algorithmic design framework for the re-assembly with weight.


#### **3.1 Motion Design-Simulation**

The motion design-simulation scope concerns the weight distribution and the geometrical rules as the driver for enabling the objects to move. This scope focuses on developing a computational method for design simulation, optimization, and motion simulation based on the physical and geometrical attributes of objects in any desired shape, weight and volume. The motion simulation and robotic fabrication constrains are integrated into a fabrication-aware design process (Bermano et al. 2017), which allows for a seamless workflow from design-simulation to materialization with robotic sequential casting method. The following phases describe the development of each step in details:

#### **On weight distribution, equilibrium, and Center of Mass (CoM)**

The main challenge in this phase is to define an algorithmic tool that generates and optimizes the weight distribution and the position of the CoM inside the hollow elements in order to achieve a stable equilibrium (Fig. 4). Two axes are determined based on the targeted CoM: A vertical axis is the equilibrium axis (EA), and the horizontal one is the reference axis of stability. A vertical zone on both sides of the Equilibrium axis is the target zone for the CoM. The generative model is based on a 3-dimensional model subdivided in discrete voxels, each Voxel demonstrate a Unit with same volume, dimension, with a certain weight in different position. We developed an optimization algorithm in Grasshopper, that shifts the CoM into the equilibrium zone. The objectives are the maximum cavity inside the object, variable wall thickness and the equilibrium of the element in any desired position (Fig. 3). The closer the CoM is located to the equilibrium axis, the more stable the building element is various mathematicians developed classification systems for geometric objects in relation to their stable and unstable equilibria. Domokos et al. demonstrate a mathematical model for classifying convex geometries by their stability (Domokos et al. 2010). Domokos et.al. Research Ranks any Convex three-dimensional shape with a numerical model based on the number of stable equilibria and unstable equilibria surfaces and curvature. Domokos et al. use their mathematical studies to understand the self-righting behavior of turtles based on the geometry of their shells (Domokos and Várkonyi 2008) (Fig. 4).

#### **On assembly sequence**

The assembly sequence in this research classifies the position of objects into three position categories based on the stability of the geometry and the CoM: (a) the rest position,

**Fig. 3.** The weight distribution inside the characters

**Fig. 4.** The weight distribution inside the characters

(b) the charged position, and (c) the locked position. Based on this classification, the building elements are addressed for assembly and interlock (Fig. 5).

The rest position is a desired stable position that the element is in equilibrium by itself regardless of assembly. The shifted CoM forces the object from any other position to roll into this position. The charged position defines any position in which the element curvatures touches the ground in an unstable equilibrium, the object is now charged with kinetic energy aiming at reaching a rest position. The interlock position addresses a situation in which the object is not in a stable position by itself but instead gets stabilized through the interlocking and the weight of the neighboring element. The global equilibrium and assembly sequence are the defining parameters for this position.

In the case study installation "*Animate Concrete*", the rest position of element #3 (named Wheel) is defined in the position when the objects lay back with a 45° deviation to the ground. (It means the hollow element will always roll back to this rest position). The element #2 rest position is defined in the vertical upright position. (it means that the object would stand up in the rest position). When Element #2 and 3# are rolled to the horizontal position (charged position), the weight of Element #1 would keep the

**Fig. 5.** The installation/performance "Animate Concrete" assembly sequence of concrete Characters

assembly together. Reversing this assembly sequence would result in all elements rolling into their rest positions after releasing element 1 from the interlock position.

#### **The motion patterns**

The motion path generates the curvature of the elements. The geometries are defined and categorized by their ability to the role (Fig. 6). The degree of curvature is an essential parameter for the classification of curvature. A computational model is developed to generate the curvature of any desired shape due to the direction and trajectory of motion. This computational model generates a smooth discretized convex developable surface (Rose et al. 2007) along the desired objects to roll. This framework allows an interactive computer-aided design method, which defines the geometry and physical attributes of any desired shape and assembly scenarios by the users.

#### **3.2 On Robotic Sequential Casting (RotoForm)**

Sequential robotic casting, RotoForm (Tessmann and Mehdizadeh 2020), is a material system that combines the motion trajectory of concrete formwork and material fluidity properties to materialize hollow-body objects. RotoForm allows for the materialization of hollow concrete building elements with a thin shell. This fabrication technique is a common method in the plastic industry for materializing large objects with equal wall thickness. Within this technique, a minimum amount of material (in liquid status) is poured into the formwork, which is just enough to cover the formwork. Rotating

**Fig. 6.** The calculation of the motion trajectory of each concrete object.

the formwork during the curing time allows for the distribution of the material over the formwork surface. Utilization of a robotic arm (Fig. 7) enables to vary the wall thickness by tweaking the rotation parameters: velocity and motion trajectory.

**Fig. 7** The Fabrication pipeline of hollow concrete building elements, robotic sequential casting (RotoForm).

The wall thickness inside the building elements is generated through several casting iterations and specific motion sequences. Rotating the robot for a longer time at a specific angle to the gravity direction results in more material curing in a particular part inside the formwork leading to increased wall thickness. The robotic trajectory algorithm determines the motion of the 6-axis robot arm and the amount of casting material for each iteration (Fig. 8). The robotic fabrication framework, controlling the material deposition, is comparable to the slicing tools used in 3d printing. The robotic casting sequences are modified to build the outer shell with even material thickness in the first and second iterations. Subsequently differentiated layer thickness is achieved through uneven movement in the following iterations. An adaptive formwork is attached to the flange of the robotic arm, which allows for direct control of formwork position and fluid flow of liquid material inside the formwork. The material used for this project is a waterbased acrylic resin polymer and High-Performance Concrete (HPC) (Mehdizadeh et al. 2022) that was developed at WIB TU Da in different layers. The amount of material to pour inside the formwork is calculated in the robotic framework according to the desired material density and total volume of each layer.

**Fig. 8.** Robotic sequential casting (RotoForm) process pouring the calculated amount of mortar inside the dynamic formwork.

#### **3.3 Motion Tracking: A Digital Framework for Evaluating the Motion**

Within this scope, the different methods are defined to determine and compare the motion of objects in various stages of design and fabrication. A rigid body simulation of the digital model is conducted and run in a seamless connection to Nvidia PhysX in Grasshopper (Fig. 9a). The rigid body simulation allows the assignment of the density, volume, and exact position of CoM for each object based on the voxel model. The resolution of voxel grids defines and computational expenses but also the precision of the simulation. A motion tracking system allows us to compare the motion results of simulation and materialized objects (Kiss, n.d.).

**Fig. 9. a** design simulation of Wight distribution inside the concrete element and the motion. The Visualization of motion tracking Data of (**b**) kinetic hollow concrete element assemblies and (**c**) the Human interaction (DDU Motion Tracking Animate Concrete 2022)

This method serves as a four-dimensional evaluation framework with acceptable accuracy. To precisely track the object motions, we used a set of "Opti-Track" cameras and the motion tracking system in conjunction with a 3D scan model of the object (Fig. 9b). We then overlaid the data from the optical motion tracking software Motive® with the 3D scan models in Rhino®/Grasshopper. This method allowed the comparison of the of the digital model with the materialized objects in motion. Furthermore, "Opti-Track" allowed the motion track of the objects and the humans (as a user) in interaction with concrete elements (Fig. 9c).

# **4 Conclusion**

Animate Concrete provides novel strategies to save building material resources by rotoforming, thereby minimizing material consumption and significantly reducing the weight of robotically precast concrete elements in response to the climate crises and the shortage of building materials. This study shows that by programming the building element and its weight distribution, concrete elements can be mobilized through the gentle touch of a human, unlocking the possibility of reconfiguring discrete building elements in the 1:1 scale. Animate Concrete introduces the possibilities of using a seamless digital workflow from design simulation to robotic fabrication and also provides a 4-dimensional motion tracking system to survey the architectural environments in interaction with human agency. The design scope of this project is to refine the modularity in architecture by assigning different behaviors to the architectural elements, turning them into Architectural Characters. The study migrated the calculation and fabrication methods from other disciplines into the realm of robotic fabrication and digital concrete for architecture.

# **5 Future works**

Future investigations should include integrating structural analysis to test the system as a load-bearing reinforced hollow concrete element. Reusing building elements requires a digital framework for discrete design with embedded optimization components. This study is in the stage of scaling up the system with larger building systems. Therefore, machinery development for the robotic fabrication set-up is crucial for the future. Developing the interlocking system based on the rolling is the essential next step to reach the strong joined assembly of building elements. The physic-informed design framework and rigid-body simulation of other software platforms powered by Nvidia PhysX is another crucial step for this research to increase the precision. Furthermore, connection details and robust edges and corner details need to be developed to ensure durability for multiple reuse.

**Acknowledgments.** The research project "Animate Concrete" is developed and materialized with architecture students' outstanding contribution at Digital Design Unit, TU Darmstadt, in S.2022. Among all students, the assistance of Arian Sadafi, Leon Witschorke, Joshua Schäfer, and Malcolm Unger impacted the development of the design simulation and motion tracking framework. The support of the students Danial Ahmad, Bingqun Liu, Luke Schuessler, Daniel Schinkels, and Jingwei Fang specially assisted our team with the robotic fabrication. The collaboration with *Tanzpol Berlin* choreographs Ashkan Afsharian, Johanna Kasparowitsch, and media designer Maxim Abrossimov had specially reinforced us for the development and realization of this project. The motion Tracking part of this project is developed in collaboration with *IAS TU Da*, Dr. Boris Belousov, and Timm Schneider. This project has been funded for research by *Fonds Darstellende Künste* Berlin by the *German Federal Government Commissioner for Culture and the Media*.

# **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Manufacturing Process of Recycling Corn Fiber, A Low-tech Materials for Modular Construction**

Hanzhe Bao1 and Zidong Liu2(B)

<sup>1</sup> University College London, Gower St., London WC1E 6BT, UK hanzhe.bao.21@ucl.ac.uk <sup>2</sup> The University of Texas at Austin, 310 Inner Campus Drive, Austin, TX 78712, USA zidong.liu.22@utexas.edu

**Abstract.** The research demonstrates a novel approach to using various parts of maize plants (leaves, fruits, and kernels) to create building materials that can be modularized for construction purposes. Corn is widely grown as an agricultural crop, but after the removal of the fruit, the remaining parts are often discarded and contribute significantly to environmental pollution. Currently, only a few companies are engaged in the recycling of maize into building materials. However, existing methods of recycling corn have various limitations such as high energy consumption, a requirement for skilled workers on-site, and extensive equipment needs. In this project, we aim to reduce reliance on equipment, skilled craftsmanship and material resources to make the design compatible with traditional building methods for low-income areas. We first analyzed the material properties of each part of the corn and found corn husks to be the most efficient for extraction. Additionally, we obtained adhesives from the waste fruit. Finally, we designed assembly units and assembled two sturdy and reliable chairs to verify the feasibility of our workflow. The low technical, equipment, and cost requirements of this material make it possible for modular construction to be replicated in local communities, thus promoting community participation and self-management in construction.

**Keywords:** Corn fiber · Recycled materials · Modular architecture · Low-tech construction

# **1 Introduction**

### **1.1 Background**

Maize is a staple cereal, having been cultivated and improved by humans for 9,000 years [1]. In 2022, over 1 billion tons of corn were produced worldwide, accounting for 5% of the world's total food production [2]. Despite its widespread use, a significant portion of the maize plant including the stalks, leaves, and husks often goes to waste through burning or illegal open landfill practices, accounting for 30% of total corn production [3]. This phenomenon leads to severe environmental pollution and resource waste [4]. It is crucial to find a more sustainable and efficient way of recycling these agricultural waste materials.

#### **1.2 Problem Statement**

The need for recycling maize waste has led to the development of two main methods for processing maize fiber. The first method involves modifying the Wood-based Fiberboard (WBF) technique to produce maize fiberboard. The second method uses maize fiber as an aggregate for eco-friendly or bio-based concrete. However, both methods have significant drawbacks [5]. However, both processes have major drawbacks.

The first approach, producing maize fiberboard, contributes to pollution during the fiber hydrolysis process [6]. It requires expensive, technically demanding hot-pressing equipment and experienced workers for the resin and fiber mixing process. Additionally, the synthetic resin used is non-degradable and the resulting products are not strong enough for structural use [7].

While the second approach, using maize fiber in bio-based concrete, improves concrete strength, reduces cost, and enhances sound and thermal insulation properties, compared to conventional concrete [8]. However, it has unstable mechanical properties due to the uneven distribution of fibres inside the concrete. In addition, concrete materials are difficult to be recycled [9].

#### **1.3 Research Aim**

This paper aims to develop an environmentally friendly and low-tech process for producing corn fiber with structural strength. In order to minimize that pollution and energy waste, we use corn husks and a small number of corn leaves as raw materials and make a bio-based glue from corn starch. This low-tech approach can also promote community involvement in construction in economically disadvantaged regions, improving the recycling rate of local agricultural waste.

# **2 Literature Review**

#### **2.1 Why Maize?**

Maize fiber has several distinct advantages compared to other types of fibers. Firstly, maize leaf fibre is the second most produced in the world, with an annual production of over 9 million tonnes [10]. Secondly, compared to other plant fibres, maize fibre is more durable and and can withstand high tensile forces [11]. Thirdly, maize fibers have a pore structure that reduces their bulking density, making them an effective composite material [10]. In contrast to fibres such as hemp and coconut, maize fibres are easier to separate. This means that in breaking down the same amount of fibre, maize fibre requires a lower concentration of lye and a lower heating temperature. It does not require particularly high concentrations of sodium hydroxide when extracted with alkaline reagents [12].

#### **2.2 Existing Mothods of Processing Maize Fibre**

Almost all existing processes require the pretreatment of maize fibre, which is generally divided into five steps: crushing materials, adding lye (10–12% sodium hydroxide), heating lye, rinsing and drying. Different levels of pretreatment is used to obtain raw materials with different fibre impurities [11].

In the existing WBF-based process, thermosetting resins (UF and MF) are generally used as binders, and the mixed material is then added to a hot press for hot pressing [7]. Fibreboard from this processing is a lightweight material; it has the advantages of being environmentally friendly, low carbon, and good thermal and acoustic insulation [13]. However, its binder is non-degradable and it produces pollution during the fibre hydrolysis process [6]. Therefore, it is essential to develop a clean and sustainable process for transforming corn fibres into valuable products to make the best use of agricultural waste. On the other side, Its products are also not strong enough to be used as structures [7].

Corn fiber can be used as an aggregate to strengthen the concrete, called Bio-based concrete. It can effectively reduce the cost of concrete and improve sound and thermal insulation properties [8]. Bio-based concrete is also more environmentally friendly and helps to dispose of agricultural waste. However, compared with conventional concrete, it has unstable mechanical properties due to the uneven distribution of fibres inside the concrete [9].

So far, no buildings have been constructed using maize fibre as a structure, and most maize fibre products are in the experimental stage. The few lightweight maize panels are similar in structure to wooden Oriented Strand Boards (OBS boards). For example, a company called Corn Board Manufacturing produces a line of corn board products. Our research tries to explore a cleaner and easier way to produce corn fibre boards that are fully degradable and recyclable with a certain structural capacity.

### **3 Methodology**

#### **3.1 Overall Framework**

Figure 1 illustrates the production process associated with maize. After harvest, the maize is processed by a local factory, where a majority of the pulpy part is distributed as food, while some is utilized as raw material for various industries. The excess waste is either collected by local farms for use as feed or for landfill purposes. However, a significant portion of the waste remains unutilized. The recycled waste discussed in this paper comes mainly from the waste from raw material processing in the local factories.

The process of utilizing corn residue is outlined as follows: 1. Assessment of corn fiber properties and extraction of shatter corn fiber. 2. Simultaneous production of biobased adhesive and material sample boards. 3. Design of assembly units. 4. Structural strength validation by making chairs. 5. Simulation of modular building forms utilizing corn-based materials.

#### **3.2 Analysis of Corn Fiber Properties**

In this step, we analyzed the fibre properties of the waste collected from local corn husks and leaves. We found corn husks and leaves are easily separated by the lye solution. Our findings align with previous studies which have shown that the husk has the highest

**Fig. 1.** The overall framework of recycling corn fiber

cellulose content among the stalk, cob, leaf, husk and ears of maize [14] and the highest water uptake suitable for separation with alkaline solutions [15]. The following is our detailed operation procedure.

According to available fibre extraction techniques, we treated the raw corn material (corn husk, corn leaves) with physical and chemical methods respectively to improve their fibrillation [16].

First, in the pre-treatment, we manually shred the corn husks into long strips 17– 20 cm long and 3–5 cm wide. In the physical method, we further mechanically crush the corn leaves to 7–10 cm fibrous silk. And in the chemical method, we add the 17–20 cm corn silk to a boiling hot lye solution for fibre separation, after which we finish rinsing and drying to obtain the raw corn fibre. The characteristics of corn husk do not require a particularly high concentration of lye (Sodium bicarbonate) to complete the separation. The pictures show the characteristics and physical properties of the various fibres, of which boiled fibre and shattered fibre are the main raw materials for production (Fig. 2).

**Fig. 2.** Comparison of fiber properties by different techniques

#### **3.3 Making Bio-Based Glue and Material Sample Boards**

The second step of our process involves extracting bio-based glue from the maize starch. This is achieved by using xanthan gum [18], which is produced from bacterial fermentation of sugars [17] obtained from maize starch. However, due to low yield in self-extraction, we have mostly relied on commercially available xanthan gum in our experiments. To enhance the water resistance and strength of the structure, we have added Polyvinyl Alcohol (PVA) and other natural resinous materials to the xanthan gum mixture.

We have conducted multiple tests to determine the optimal combination of materials for the bio-based glue. These tests involved using different materials, ratios of xanthan gum and PVA, and even combinations of maize kernels and binders (Fig. 3).

The final blend of xanthan gum and PVA, combined with mechanically crushed maize fibres, resulted in boards with good mechanical properties, water resistance, and ability to withstand certain pressures.This mixture of xanthan gum and water in a mass ratio of 1:10 and aqueous xanthan gum to PVA in a mass ratio of 1:1, together with extracted maize fibres and mechanically crushed maize fibres (0.5:1 mass ratio), offers the desired properties for a sustainable and environmentally friendly bio-based glue. In other experiments, xanthan gum alone has been used as a load-bearing material and can be re-moulded with water, but its water resistance is not as strong as the blend of xanthan gum and PVA.

**Fig. 3.** Different combination of maize kernels, fibres and binders

#### **3.4 Modular Building Form Simulation Using Corn Materials**

Our team designed a series of shapes of assembly units (flat, curved, minimal surfaces, prisms) and test their structural performance relatively. All shapes of assembly units are made of xanthan gum and maize leaf fibre (1:5 mass ratio). To evaluate their structural performance, we conducted preliminary assessments of various properties such as strength, weight, flexibility, porosity, and recyclability. The results showed that most shapes had low weight, high flexibility, and were less likely to break, but more susceptible to deformation under vertical pressure. Flat shape of fibres board are tough and have a nice texture, but they can easily bend and can't handle high pressure. Curved and minimal surfaces shape fibers are better at handling pressure, but they're not as tough. Prism-shaped fibers can handle the most pressure and are easy to make with molds (see Fig. 4).

Additionally, we explored the architectural forms of each unit shape and assessed their compatibility with the glue mixes discussed in the previous step. Our team also experimented with various methods of joining parts to optimize assembly. After evaluating all options, we selected the trigonal prism as the preferred design for future development.

**Fig. 4.** Samples of different shapes

# **4 Case Study-Chair**

#### **4.1 One-piece Molded Chair**

To validate the structural soundness of the trigonometry unit produced by this work flow, we designed and assembled two chairs, each with different compositions and molding techniques. The first chair was comprised of a 50:50 blend of mechanically crushed maize fibre and chemically extracted maize fibre, while the binding solution consisted of a mixture of xanthan gum solution (1:10 aqueous solution) and bio-plastic (CA, Cyanoacrylate-structural hybrids) in a 6:5 mass ratio [19]. Regarding mould plasticity, the team used CNC cutting technology to cut the Foam Board. The first chair was made with a one-piece infusion mould. The mould of the chair was cut longitudinally into eight pieces to obtain the rectangular shape of each piece that needed to be hollowed out, and the CNC technology was used to remove the parts that needed to be hollowed out on the eight different PVC Foam Boards. The eight PVC Foam Boards were then glued together and moulded. After 36 h of drying, the moulds were removed, and the first chair was made.

#### **4.2 Modular Chair**

For the second chair, we used 65% mechanically crushed maize fiber and 35% chemically extracted maize fiber as the material and reduced the use of bio-plastic, which is not environmentally friendly, by replacing part of it with PVA in a ratio of 5 (PVA):5 (bioplastic):12 (xanthan gum solution). To improve the production process, the research team aimed to produce the chair in a modular form. As the chair can be divided into 12 finished trigonal components, we divided it into two groups, one consisting of four trigonal pieces spliced together to form the back of the chair and the other connecting two back trigonal pieces. Polished corn cobs are used as the dowel to connect these components. CNC was used to make these 6 moulds in the meterial of PVC Foam Board (Fig. 5). In fact, many of the moulds for the components are the same and they could eventually be reduced to 2 different moulds. The finished components were then joined together using polished corn cobs and perforated joints after 36 h of drying.

**Fig. 5.** Proceing of making

#### **4.3 Use Test and Comparison**

Both chairs boast strong and durable design, able to comfortably support the weight of an average adult (Fig. 6). However, the first chair is susceptible to cracks and moisture due to the different moulding materials, which are caused by the high level of chemically extracted fibres, and the lye has damaged the structure of the fibres to some extent. In terms of moulding, the first chair is much simpler and quicker, and the mould saves space. However, the integrated design makes it challenging to remove and dispose of the mold. On the other hand, the second chair may take longer to mold but its design allows for easy removal and reuse of the mold.

**Fig. 6.** Two chairs and imagine picture

#### **5 Conclusion**

This paper verifies the feasibility of using agricultural waste corn to create modular building materials. After performance tests via making two chairs, we found that the material has potential as a building surface and interior material (Fig. 6). Compared to existing recycling processes, the process presented in this paper is more cost-effective, requires less equipment and has a lower environmental impact, enabling fast construction and community involvement.

However, due to research condition constraints, there was a lack of experimentation in terms of architectural dimensions. The size of the model was only up to the size of the furniture, Besides, no rigorous testing methods were used for the mechanical testing of the samples. In addition, as no long-term tests have been carried out, it is not known whether the chairs will swell or deform when exposed to moisture over a long period of time.

Despite these limitations, the proposed method has the advantage of being less expensive, requiring less equipment and faster, making it ideal for local self-construction projects. The technology is open-source, providing the potential for communities to develop their own local building material and construction companies through a peer-topeer collaboration model [20]. In this model, the government grants land development rights to the community for public welfare, the local construction materials companies recycle corn waste from farmers at a low price, employ local residents to manufacture building components and build them locally [21], and the architect establishes a platform to provide technical support [22]. By promoting a participatory, co-productive housing framework, architects are converted from former environmental designers to platform system providers, and community residents, formerly pure consumers, are converted to producers [23].

# **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Membrane-Based Modularization in Prefabrication System Design as a Strategy in Emergency Buildings**

Yunsheng Su1, Xihan Cheng1(B) , and Jiaxing Li<sup>2</sup>

<sup>1</sup> Tongji University, No. 1239, Siping Road, Shanghai 200092, China {suyunsheng,chengxihan}@tongji.edu.cn <sup>2</sup> University of Melbourne, Grattan Street, Parkville, VIC 3010, Australia

**Abstract.** This article is about the application of air film-based modular prefabrication system design strategy in emergency buildings. To determine the typical and essential behavior of membrane structures, this paper reviews and compares relevant theories, experiments, and simulations on architectural performance, mechanical properties of membrane materials, and membrane structure performance. In addition, this project tries to use the robotic arm spraying technology to study the strategy of trajectory generation. First, import the model shape into the software for multiple trajectory simulations, verify and optimize the injection path program through the software, observe the actual injection trajectory, and finally select the final Program. The resulting vectors are also relatively balanced, avoiding the over-concentration of the top curve. The advantage of this method is that it can reduce the material on the top and make the material distribution more uniform, which is a more economical and reasonable way to obtain the trajectory.

**Keywords:** Emergency building · Prefabrication system · Modularized design · Inflatable membrane · Robotic spraying

# **1 Introduction**

In recent years, the inflatable membrane structure has diversity of modeling, excellent architectural, structural characteristics and suitable economy, etc. It has been favored by people in many fields due to incomparable advantages over other traditional buildings. For example, in emergency medical treatment, membrane building can be used as a temporary building near the recent outbreak. In the case of limited manpower and material resources after a disaster, the designed air flotation membrane package can be transported to the site (shown in Fig. 1). The local construction team only needs to inflate and spray gypsum, cement clay and other materials on the surface. The arched inflatable membrane structure can be relocated after the peak period, and a permanent shell structure is formed by spraying building materials on the surface of the inflatable membrane, and the sprayed shell structure can be further transformed into a lawn building as a public facility. The construction of membrane-mold combination is the combination of intelligent design and automatic control construction. The goal is to achieve intelligent improvement and energy consumption reduction in the whole life process from architectural design, building construction and subsequent practical operation. In general, the overall performance of membrane structures is a multidisciplinary topic consisting of material selection, performance control, and sustainability.

# **2 Research Background**

#### **2.1 Analysis of Emergency Building Design**

Since the 2019 pandemic, modular buildings have clearly influenced the way people around the world live, work and play. Since the emergence of new standards for social distancing, the use of emergency buildings has received increasing attention from the construction industry. From the current situation, governments have developed quarantine policies that restrict the movement of labor, materials and equipment needed for construction activities. As a result, construction stop-work, schedule delays, supply chain disruptions, rising labor costs and skilled labor shortages have become major obstacles to the development of the construction industry.

The membrane modular building as an emergency building strategy can solve the existing problems. As shown in Fig. 1, a hospital in Nur Sultan, the Republic of Kazakhstan, temporarily set up a Huo-Yan Laboratory in response to COVID-19. The building was designed, manufactured, transported within 10 days and assembled on site to quickly support scarce medical facilities. The factors of strong function, high plasticity, free form and sustainability of air membrane architecture have attracted wide attention. Its popularity mainly depends on easy installation and excellent structural and architectural performance [1, 2].

The evaluation of the architectural performance of membrane structures is essential for the structural performance and the corresponding practicality, because the applications of these buildings are mainly placing where people gather. The safety and applicability of the two aspects of the building structure need to be satisfied, to achieve these two aspects of the basic factors of the standard is the appropriate material and the corresponding mechanical properties produced by the different effects and loads. It is very important to correctly use the relevant mechanical characteristics and properties to analyze the building structure for membrane buildings, especially for places with high human flow, emergency buildings, etc. Therefore, the evaluation of membrane architecture is also studied from three parts, including building performance, material characteristics and structural performance.

Nowadays, nonlinear building and modular building gradually attract the attention of the construction industry. Along with the development of computer technology, robotassisted manufacturing has been introduced in the field of automated production for many years. Ulrich Muther's construction firm has applied shotcrete technology to various shell structures and bobsleds [3] (Fig. 2). Concrete is applied in layers and annular formations using wet spraying techniques without the use of additional formwork [4]. Most nonlinear building forms are manufactured by CNC machine tools [5]. Due to the large volume of building products and high load-bearing requirements [6], simple machine tool processing is not conducive to the rapid production of large building components and the use of concrete building materials [7], and the production of buildings still follows the traditional manual process. Taking advantage of the new possibilities of digital planning, the construction industry has demonstrated the potential of realizing free-form architecture. Currently, there are two types of mainstream concrete 3D printing construction: on-site printing and prefabricated assembly: On-site printing is a mature technology with no segmentation issues; however, prefabricated assembly raises more concerns; and it is expected that more large-scale 3D printing funicular spatial structures will emerge in the near future [8].

The experiment in this paper focuses on the combination of flexible robot components and accurate and reliable processing system, which might be redeemed as an innovative attempt in the field of construction industry. The facility will have a unique function and will have a significant impact on building manufacturing in the future.

**Fig. 1.** Huo-Yan laboratory

# **2.2 The Experiment Design idea of this Project**

Membrane modular composite intelligent additive construction is a construction method with modular manufacturing, flexible material application and automatic control as the core: We form a modular solution through the water and electricity needs in the prefabricated building, handing the function of building space envelope to lighter and more flexible materials. Based on the temporary building built with modular components and flexible skin as the core, flexible skin as the template, controlled by mechanical arms and sprayed with curable liquid materials, the final hardened building shell is formed to complete the rapid construction of the building.

**Fig. 2.** Oberhof construction of the bobsleigh using shotcrete methods

The core concept and value of this construction method is to provide a construction solution that is easy to transport and construct: Flexible skin provides a solution for construction with an ultra-high compression ratio, greatly reducing the space, weight and cost required to transport construction materials. The design and construction can be combined with local materials, which further reduces the difficulty of logistics transportation and off-site construction. It is especially suitable for the construction needs of sudden climatic disasters and natural disasters. Compared with traditional construction methods, the transportation volume is smaller, and the requirement and application of local materials also make this construction technology more in line with people's imagination and needs for extraterrestrial construction [9].

# **3 Experiment Process**

#### **3.1 Structure and Texture Design**

The three-dimensional size of the gas film structure is 4300mm in length, 3200 mm in width and 3000 mm in height. Modeling was carried out by Rhino software; the structural parameters of the film were inputted, and the spraying trajectory was designed (Fig. 3).


through the predetermined position, and restarts spraying at the set position. Some methods of spraying the air film with doors,Windows and other obstacles are explored.

4. Two-point spraying of the robot arm is proposed, and the placement scheme of the robot arm that uses the mid-point to change the spraying direction solves the limitation of the physical spraying Angle of the robot arm to the greatest extent without the slide rail, which lays a certain foundation for the future research on the spraying of the robot arm in a narrow and long space.

**Fig. 3.** Spraying trajectory simulation of manipulator

#### **3.2 Aerated membrane construction**


reached and the cylinder basically reaches vertical. The aeration molding process is completed, as shown in Fig. 4. Then continue to inflate, the gas film inside and outside to maintain a constant pressure difference.

**Fig. 4.** Inflatable film display

# **3.3 Polyurethane Construction**


#### **3.4 Concrete Construction**

Concrete shall be mixed according to the strength requirements of the design on site, and concrete spraying can be carried out only after the spray gun and pump pipe are fully wetted.

It is necessary to ensure that the air compressor can provide enough power during injection, because the high-speed pneumatic consolidation of shotcrete has a positive effect on its durability [11]. After meeting the spraying requirements, spray is carried out in two layers according to the predetermined spraying trajectory, each layer is 1.5–2 cm thick. 24 h after each layer of concrete is sprayed, spray the next layer of concrete according to the same process.

# **3.5 Construction Deformation Monitoring**

# **3.5.1 Monitoring Method**

Before polyurethane construction, marking points are pasted on the outer surface of the gas film. After each spraying process is completed, the three-dimensional coordinates of the cross center of the marking points are observed by the total station, and the deformation of the gas film is represented by the change of the coordinates of the marking points.

# **3.5.2 Marking Points and Their Layout**

The marker is a solid black circle with a diameter of 3 cm and a white cross in the center. Make sticky stickers in advance and paste them on specific positions of the gas film according to the marking point layout scheme on site.

### **3.5.3 Marking Point Paste and Total Station Measurement**

After the gas film is fixed and before it is inflated, clean the dust and water on the outer surface of the gas film with a dry rag to ensure that the marks can be firmly pasted. According to the marking point cloth set the square case, along the cutting slit of the air film (the square direction of the bus line) use tape measure to determine the position and paste the marking point.

The total station is used for measurement. The fixing and leveling of the total station are completed before the polyurethane construction, and the marking points are checked to see if they are firmly pasted. The measurement of the total station is completed after the completion of each process of the polyurethane construction.

# **3.5.4 Monitoring Results**

– Deviation evaluation from design to construction

In this construction process, there are three times that may lead to the deviation between the final result and the design scheme, which are: the deviation between the design model and the inflatable film, the deviation between the inflatable film and the spraying polyurethane, and the deviation between the design model and the spraying gas film. Since the inflatable film model is generated by SLR shooting, the accuracy is low, so the bias evaluation is mainly carried out from the last Angle. Point cloud analysis is carried out between the model generated by 3D scanning and the original model designed. It can be found that except for the ground, the overlap rate of the whole model and the original design model is very high, and the deviation from design to construction is very small.

– Deformation evaluation of gas film structure during spraying process

The gas film was sprayed twice in total, and the deformation of the two spraying processes was obtained from OpenCV data. The first deformation is large, about a centimeter or so, while the second almost no deformation.

It is known that after the first spraying, the second spraying after hardening of the structure will hardly cause the deformation of the structure again. Regarding the deformation of the structure, the best way to minimize the deformation when spraying polyurethane might be replacing the material of the film with a less elastic one; reducing the thickness might be the next best.

– Stability evaluation of spraying process

According to the data obtained from OpenCV monitoring during the first spraying process, it can be found that the target point has a certain displacement in X, Y and depth during the spraying process, but the displacement is not large. Combined with video analysis, we believe that the spraying process is very stable, and the absorption and release of heat in the solidification process of materials is the main cause of target displacement. The spray process is very stable.

– Spray thickness uniformity evaluation

In the process of spraying, the nozzle keeps good discharge condition, which is basically the same as the design track of the track group. The tracks on the lower side of the structure are dense and thick, while the tracks on the upper side are sparse and uniform and reasonable in thickness.

# **4 Findings**

#### **4.1 The Operation of Spraying Trajectory of Manipulator**


the connection and installation of components, it is necessary to reduce the material of the components of additive molding. The reduction process is best carried out in situ and should be carried out immediately after the completion of additive. To facilitate later installation, it is necessary to insert steel bars or conduits in advance in the enhanced member when the strength is not high.

4. In more architectural projects, in the addition of walls, columns and outer surfaces, architects can make use of the characteristics of mechanical arms for rich curved surface design and accurate manufacturing. The floor and roof can be processed longitude-wise first, then turned over by hoisting and combined with the wall column. With the large-scale popularization of concrete processing system by mechanical arm, more nonlinear buildings with rich shapes will appear in the future.

### **4.2 Defects of Spraying Technology**

The spraying trajectory of the manipulator has great influence on the overall quality. There are several guides on best practices for sprayed concrete (e.g. ACI 506R-16 2016 and EFNARC 1999). Virtual reality has been effectively used to train water gunners before they go underground. Poor spraying techniques can result in the following defects (see also Fig. 5).


The prefabricated starter rod unit simplifies the joint by optimizing the spraying trajectory and avoiding inconvenient geometry. Strips up to 40 mm were successfully sprayed, but this was very difficult, especially where strips overlapped or crossed. The latest technology aims to reduce the scope for human error. Various range-measuring devices (for example, laser rangefinders such as TunnelBeamer2 or photogrammetric devices such as

**Fig. 5.** Cases of spraying failure

DIBIT) are used to examine the profile of shotcrete. TunnelBeamer is currently a system that can be used interactively to check profile and thickness during the spraying process. That's the advantage of the device, over the fact that it can only take field measurements. The advantage of devices like DIBIT is that they can inspect the entire surface, but you have to stop working to complete the measurement. In emergency construction, such as tunnel construction, the typical accuracy of these systems is ±20 mm, which is sufficient considering that spray tolerances are usually ±15 to 25 mm.

# **5 Results**

The model established by Rhino software is imported into the finite element calculation software ANSYS for structural calculation. The concrete shell adopts the layered definition of shell element. The specific material parameters are shown in the Table 1.



As shown in the figure, we analyzed the static load and wind direction load of the model structure through Ansys software, and the results were displacement, maximum tensile stress and compressive stress respectively. Through the comparison of the results, we found that the result of the wind direction load in the y direction is the largest, so we mainly analyze this load condition (Table 2).

According to the Rule NO. 3.2.11 in the structural code of "Specification for design of reinforced concrete shell structures" (JGJ 22–2012), In the limit state of normal use, the deformation of the edge members should be checked unless there are special requirements. Under the standard combination of loads or quasi-permanent combination and the long-term effect of loads, the deflection value should not be greater than 1/400 of the span when the span is greater than 7 m; it should not be greater than 1/250 of the span when the span is not greater than 7 m. The orange part in the figure shows the most deformed area of the model, where the maximum displacement is 1.75 mm. In our case, the span is known as 4.3 m. Therefore, following the guide of structural code, the displacement value is within the safe range (Fig. 6).

According to the data provided by the material supplier, the maximum tensile value of the concrete is 6 MPa, and the result of the model is 2.3 MPa, so the structure is safe (Fig. 7).

The blue area of the picture shows the largest compressive area in the model, where the maximum value is 2.55 MPa, and the maximum compressive limit of this material is 60 MPa, so the structure is safe (Fig. 8).

# **6 Discussion**

The robot aided shotcrete technology makes the concrete applied uniformly and realizes the heterogeneous concrete structure. The surface structure of the finished shotcrete is rather rough. Additional finishing steps will be necessary to achieve smaller surface tolerances. It is possible to program the motion to build the desired shape.

In addition, the process can achieve a low rebound rate of about 8%. Despite good initial performance, quality changes were observed during concrete spraying. This is due to the use of a pumping system without a control loop, resulting in uneven mass flow of concrete. This situation has a large effect, and the effect of the robot's path parameters on the application quality cannot be observed.

# **7 Conclusion**

The application process of shotcrete using a mechanical arm has shown that it leads to the application of heterogeneous materials. This problem must be solved in the production of parts, or at least considered in the design of the workpiece for the process. Beyond these traditional applications, there has been little exploration of new applications, despite the advantages and potential that shotcrete brings. The main reason for this is the traditional manual spraying technology. To achieve good results, a high level of experience in the movement and use of tools and the right mix of materials is required. Often too much or too little material is used, and sometimes the concrete composition of the coating is not particularly uniform. The attainable quality in terms of geometric accuracy, as well as visual and tactile surface quality is significantly lower than those of poured concrete. Therefore, the great advantage of mechanical spraying process is the flexibility


**Table 2.** Structure analyses results

**Fig. 6.** Maximum displacement

**Fig. 7.** Maximum tensile stress

of application, especially in the case of emergency building in line with the structural safety to meet the characteristics of high efficiency and low cost.

In summary, our results show that robot-assisted concrete spraying has great potential in the generation and manufacturing of free-form surface concrete parts. For further research, concrete pumping systems with more complex control loops must be used to improve the quality of concrete construction. The system will be implemented in a demonstration research facility and additional experiments will be conducted to generate manufacturing optimized path movements.

**Fig. 8.** Maximum compression

# **References**


432 Y. Su et al.

11. Jolin, M., Melo, F., Bissonnette, B., et al.: Evaluation of Wet-Mix Shotcrete Containing Set-Accelerator and Service Life Prediction (2015)

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Customized Knit Membrane Deployable Hyperboloid Tower**

Virginia Ellyn Melnyk(B)

Virginia Tech, 201 Cowgill Hall, 1325 Perry St, Blacksburg, VA 24061, USA vemelnyk@buffalo.edu

**Abstract.** Deployable structures have become increasingly popular due to their ability to transform from a compact form into a larger structure. They are also typically lightweight, resulting in a lower carbon footprint than heavy permanent building methods. These structures are popular within the field of architecture, as well as in robotics, aerospace engineering, and other fields.

This paper explores the design and development of a deployable hyperboloid structure with a connected knitted membrane. The knitted material is specifically designed to stretch and fit the transforming geometry of the hyperboloid. This is achieved by manipulating the types of yarn used across the membrane, as well as the number of short rows in the knit material, to create a more specified material. The design for this material was developed using Rhino3d and Grasshopper. Throughout the design and fabrication process, there was a feedback loop between the digital design models and physical material test samples to ensure that the knit would fit the final hyperboloid structure. The result is a two-meter-tall structure when upright and a two-meter-diameter circle when collapsed, with a knit membrane that supports the structure and transforms seamlessly by stretching and shrinking to the different shapes of transformation.

**Keywords:** Deployable structures · Hyperboloid · Transformable structures · Knitting · Computational design

# **1 Introduction**

This paper provides an exploration of the design and development process of a deployable hyperboloid structure connected to a specified knitted membrane. It references precedent examples of deployable and lightweight structures, as well as examples of structures that utilize knitted membranes.

Compared to woven materials, knitted membranes offer elastic properties that are advantageous in this type of transformable application. The design pattern for this specified knitted membrane is developed using Grasshopper and Rhino3D, with a back-andforth process between testing materials and digital simulations. The resulting membrane is knitted manually on a domestic Brother knitting machine using two different yarn types. The outcome is a two-meter-tall structure when erect and a two-meter-diameter circle when collapsed, with a knit membrane that supports the structure and seamlessly transforms by stretching and shrinking to the different forms.

#### **2 Deployable Structures**

Deployable structures are structures that can easily be assembled and disassembled quickly, making them ideal for temporary or mobile applications. Dating back to nomadic cultures that used deployable and portable structures for shelter (Rivas-Adrover 2015).

Folded and jointed structures are often made of linear members and use a membrane or fabric to create an enclosure (Guest 1994). Deployable structures find use in a wide range of applications, such as military operations, disaster relief efforts, events and festivals, as well as space exploration (Rivas-Adrover 2015).

Deployable structures are also typically lightweight, utilizing textile membranes for enclosure, resulting in a lower carbon footprint than heavy permanent building methods.

#### **2.1 Deployable Structures Examples**

Some of the best examples of deployable structures come from Buckminster Fuller's designs for geodesic domes. These structures consist of interconnected triangles, which are repeatable and easy to assemble and disassemble (Buckminster Fuller 1982).

More recently, Chuck Hobermen has developed designs using scissor structures, which can expand and collapse needing no assembly on site. Most known for his Hoberman sphere, but he has developed many geometries including domes and arches (Kronenburg and Klassen 2006).

The advantages of expandable structures is that they require less assembly on site and can be assembled off site and collapsed down to a small form for transportation and brought to a site and expanded to their larger deployed form.

#### **2.2 Hyperbolic Structures**

A ruled hyperbolic lattice structure is mathematical construct in three-dimensional space that consist of straight lines intersecting each other (Beckh 2015). The Hyperbolic lattice geometry is created by dividing a bottom and top circle into equal parts, with the number of divisions determining the number of lines required to construct the hyperbolic form (Maden 2015). Moreover, there are a few parameters of note to design a deployable hyperbolic structure. First is the length of the lines. Second is the number of lines, which is determined by the divisions of the circle, and third is the rotation angle of the lines.

In the example shown here an 8-division circle creates an octahedral hyperbolic form. The lines shifted clockwise and counter-clockwise directions along the division nodes define the crossing lattice structure.

It is important to note that the shift angle effect the resulting shape of the hyperbolic structure. For instance, a 0° shift will produce vertical lines and a cylindrical form, while a 180° shift will not result in a hyperbolic shape, but in two cone shapes (Maden 2015). Therefore, depending on these parameters there are only a limited number of possible shifted angles to create a desired hyperbolic shape. See Fig. 1.

When using consistent line lengths, the size of the circle must change based on the number of shifts as well. See Fig. 2. This results in more crossing of the lines as well as a more apparent hyperbolic curvature in the shape.

**Fig. 1.** Different hyperboloid designs based on different shift positions from 180° to 0°.

**Fig. 2.** Different hyperboloid designs based on different twist shifts between the bottom circles to the top with consistent line lengths.

Lastly when looking at deployable hyperbolic forms, the line length remains the same while the circle sizes expand and contract causing the structure to transform from an erect position to a collapsed circle position. See Fig. 3.

This collapsing form is advantageous for a deployable structure as there are two collapsed shapes where the form becomes flat and small for transportation. It can be collapsed fully in the erect position as a vertical set of lines or as flat circle. This can be advantageous for shipping and transporting in a flat form. While once at a desired location can be deployed into the more three dimensional positions.

**Fig. 3.** Consistent line lengths and shifted angles resulting in a change in base circle size.

# **3 Knitted Membrane**

To create enclosure on many folding and scissor structures membranes are used. Due to the transformative shape of the hyperbolic form, a membrane, which has some flexibility to stretch, was necessary. Knitted fabrics are formed by interlocking rows of yarn in loops. The interlocking rows of loops result in a flexible and stretchy material, unlike woven material, which are constructed of a warp and weft, made from separate yarns, and only elasticity is dependent on the material properties rather than the structure.

The stretch in knitted material is due to the structure of how the loops are interlocked. When a knitted material is stretched, the loops in the material are pulled and the yarn shifts and slips, causing the loops to elongate or shrink depending on the forces. Once the force is released, the loops shift back to their original relaxed state (Samuel Poincloux et al. 2018).

Like other elastic materials, knitted materials have a positive Poisson Ratio, meaning that when stretched in one direction they get thinner in the other. This property is very useful specifically when looking at the type of shape shifting that the hyperbolic form takes as it is collapsed from erect position to a collapsed position.

The elasticity of a knitted material can also be adjusted by changing the size and tension of the loops. For example, a material with larger and looser loops will be stretchier than a material with smaller and tighter loops.

Furthermore, different yarn types will inherently have their own elastic properties depending on the materials it is made of. For example yarns such as cotton have very little stretch while acrylic and wool have more amount of stretch. However, synthetic yarns using nylon and elastic materials can have a quite lot of stretch to over 4 times their resting length.

Also unlike woven materials, which use a warp and weft, knitted materials can also be shaped into different specific geometries by adjusting the number of stitches per row through narrowing or widening. As well as using a technique called short rows, which do not go all the way across the knit row to make different number of stiches on one side of the material compared to the other. See Fig. 4.

Overall, knitted material is a versatile and flexible material that can be customized to suit a wide range of applications. Its stretchiness and elasticity make it particularly useful for applications where flexibility and movement are important.

**Fig. 4. a** Normal Kitting **b** Narrowing, **c** Widening, **d** Short rows.

# **4 Design Process**

The process for designing a prototype for the deployable ruled lattice hyperbolic tower used a process of computational design modeling to design and prepare a knitting pattern to be fabricated. When working with knit materials, the dimensions of a small swatch sample of that material are used as data and measurements to be input into the digital modeling to calculate the resulting number of stitches need for a full pattern of the final design to be fabricated.

# **4.1 Grasshopper to Model Hyperbolic Form**

Based on the pervious mathematical understanding of hyperbolic geometries. The design and analysis of the desired hyperbolic form was done in a Grasshopper model which could be transformed using set of determined parameters was created. The desired model would have consistent line lengths and the circle diameter and height would change as the angle between the lines to be adjusted to create the deployment. See Fig. 5.

**Fig. 5.** Grasshopper model of a deployable ruled hyperbolic structure.

From this series of modeling a few decisions could be made on which design to build physically. First was determined which rod length to use. Based on the purchasing off the shelf rods. Standard rod dimensions were selected for the propsoal, which uses a rod length of two meters.

Secondly the number of rods to be used had to be determined. In the simulations it was shown that the more rods used the wider the resulting final diameter of the clollapsed circle would be. To keep this constraint somwhere in the middle range where the circle would not get too large compaired to its height. A decision was made to use 20 rods. This means each circle was divided into 10 divisions.

The final variable in the hyperbolic design was to select the number of rotations for the lines as difined the angle of rotation. Given a dechahedral hyperboloid, the possible shifted lenghts could be 36, 72, 108 or 144°. The decision was made to use a 108° shift so that the rods would cross 5 times and would create a curved hyperbolic shape as the form is collapsed. See Fig. 6. This would add a certain amount of deformation in the knit material to be developed.

**Fig. 6.** Various positions of a deployment of the decahedral hyperboloid design with 10 divisions, 2 m lines, and a rotational shift of 108°.

#### **4.2 Grasshopper to Model Membranes**

The design for the knitted membrane surface was based from analysing the shift in size of the geometry from start position, as erect, to the end position, as completely flat. This was done by extracting the triangle meshes at the various stages in the design. When fully erect the size could reach almost zero so it was determined to use an almost erect state as the base, which would be more realistic given the material thickness of the rods.

Each face of the geometry could be broken down into isosceles triangles between the rods. In the erect state, these triangles would be very skinny where the height would be much more than the width of their base. Meanwhile in the flattened end position the triangles would be an obtuse isosceles triangle. The amount of deformation between these triangles was calculated to estimate the material transformation needed. See Fig. 7.

#### **4.3 Knitting Design**

To develop the actually dimensions of the knitted material to fit these the hyperboloid the size of the triangles were measured at the different states, and compared with sample knit materials. Different yarns were tested and stretched in each direction. These swatch samples used a base dimension of 60 stitches along 60 rows to get an estimate of a sample stitch size when stretched height wise and width wise.

Material swatches were made with different stitch lengths and different yarns. Each of the different material swatches were measured to see the amount of transformation, which occurred. The amount of deformation in the middle triangles of the geometry is much less that of the triangle at the top and bottom of the structure. This meant that there was a need for a larger deformation in that region of the membrane design.

**Fig. 7.** Triangular panels at different stages of deformation from the proposed hyperbolic structure in Grasshopper.

The decision was then to use a 2/28NM acrylic yarn. 2 meaning 2 ply which is that 2 single yarns have been spun into 2 plies. Two-ply yarn is better than single ply because the ply twist offsets the torque inherent in a single yarn. 28NM meaning that 28 m of single ply yarn weighs 1 g. This is a common and readily available yarn. The color red was chosen to make it contrast the other yarn, which would be used. From the knitted swatch samples of this yarn a decision to use a mid-range stitch size of 5 was decided. This would allow for a decent amount of elastic properties in the knit. In the samples a tighter stitch length such as 1 did not allow for enough stretch. Meanwhile a stitch length of 10 would have resulted in too loose of a knit, given the yarn weight, causing lots of dropped stitches as well as too much porosity and transparency.

The shaping to create a circle from the knit design utilized short rows of nylon elastic yarn. The Nylon Elastic yarn used was LP-20C which is heavier weight and has a elastic core with nylon threads spun over it. In the swatch samples due to the yarns properties to stretch during knitting and shrink back to a relaxed state after, the swatches at small stitch lengths became very tight and did not stretch much. Although the thinness of the yarn a stitch length of 10 allowed it to still knit tightly and have quite a bit a stretch still available after knitted.

The amount of elastic material and pattern for this nylon material was calculated from the surface of the flattened geometry in grasshopper to determine how many extra rows were needed. This calculation created an irregular pattern which was then simplified in to a regular pattern, using Grasshopper, that would be more repetitive and easier to knit. Here by each line in the final pattern design is represented by a partial movement of the yarn carriage to the left and right. The blue lines represented the acrylic yarn and the pink lines the nylon. See Fig. 8.

#### **5 Fabrication and Result**

The final fabrication and construction of the two meter tall hyperbolic tower was divided into construction of the lattice structure made of 2 m by 8 mm thick Glass Fiber Rods (GFR) strut structure and by attaching the specifically designed knit membrane. In addition, using very few materials for the final structure the process of fabrication took a

**Fig. 8.** Grasshopper development of short row and yarn patterns from the circular surface to knit short row design.

few days to fabricate the materials and attach it to the lattice structure. Once fabrication was complete, the deployment was a matter of pushing on the structure to transform its shape.

#### **5.1 Knitting Fabrication**

The knit material was constructed out of the simplified repeating pattern. The design for this pattern was printed on a piece of paper to help guide the knitter while making the design. Which needed little more information than how many short rows to make each length and when to switch materials.

Knitted on a Brother Ameno kh836-e domestic knitting machine, most of the design was to be manipulated by hand. The knit started with four rows of red acrylic yarn knitted with stitch size 5. To create the short rows several needles were moved to the hold position and the knitting carriage set to allow for partial knitting. The stitch length is then changed to 10 for when to knit the elastic yarn, and the yarn is knit for 4 rows at time across the partial needle bed.

The resulting fabrics was a total of 1560 rows in each panel. In addition, it took a total of 5 h to knit each panel. Because of the size of the knitting machine, it had to be knit in two panels, one for the top half and one for the bottom half. The two panels were then hand sewn together after being attached to the frame.

#### **5.2 Result**

The final structure was able to easily stand on its own in a balanced state both mostly erect position and once collapsed into a circle shape. The knit material was well calculated that when it is erect it is stretched height wise and has very little wrinkles and the elastic yarn is constricted into a small stripe which also causes a sort of pleating effect in the material. When the structure is pushed down the knit begins to stretch more width wise and the white elastic yarn begins to stretch more than the red acrylic yarn, exposing the pattern of the short row design to make the expanded circular shape.

This stretch in the yarn also creates more transparency as the holes from the loops open up and become wider allowing more light to be able to penetrate through. While when the knit is the more erect position, the holes are smaller and stretched vertical creating more enclosure and less light to penetrate.

Moreover, the tension in the yarn and the sticks provides the structure with balanced compression and tension and when the design is compressed to a certain height, it holds its shape at that height and position. Creating different stationary positions for the design rather than the expected possibility that the design would be more relaxed in one state. See Fig. 9.

**Fig. 9.** Completed tower design fabricated design at various different positions.

### **6 Conclusion**

The final design proves the successful prototype of the ruled hyperbolic deployable structure. The advantage to which it is very light weight and can completely collapse into a flat circular form or a vertical linear form to be easily transported. The design is also lightweight and can deployed by a single person at this scale. The result was of a highly collaborative and iterative design process, which made use of advanced modeling techniques in Rhino3D and Grasshopper. Using material sampling to determine the different size and types of materials to be used. Furthermore, using that information to develop a pattern that could be knitted based on the defined needs for the specified material transformation. The materials design were specifically generated to achieve the desired shape and performance characteristics to transform from a small vertical form to a flat circular form. The hyperboloid lattice structure, which was connected to a knitted membrane, was designed to also be lightweight and have specific qualities of how the number of struts and crosses needed to provide support and form.

The resulting structure was two-meters tall when upright and 2-m round when collapsed, and the knit material was able to transform with this shape change by stretching and shrinking. The success of this prototype demonstrates the potential of advanced modeling techniques and materials to push the boundaries of what is possible in architectural design and construction.

With further research and development, such structures could have a wide range of applications, from temporary shelters to large-scale buildings that transform as well as possible infrastructure. Furthermore, the study into deployable structures with knitted membranes provide many opportunities to provided enclosure and to be transformable and stretch with the shape-change of the geometry while it is in different positions.

# **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Interlocking Units for Robotically Fabricated Architectural Structures**

Myles B. Sampson III(B) and Larry Sass

Massachusetts Institute of Technology, 77 Massachusetts Ave, Cambridge, MA 02139, USA msamp@mit.edu

**Abstract.** Throughout this paper, we introduce a novel design-driven method for the robotic assembly of unit-based structures. The goal of this research is to establish a method to robotically fabricate discrete structures, using pick-andplace robotic manipulation and customized 3D-printed geometric units. Thus, the methodology allows for the bespoke discretization of architectural solid models into interlocking architectural units. Investigating how design can reduce error in the robotic fabrication process, a significant feature of this research is the application of mechanical coupling for the creation of self-interlocking geometry. This method is able to correct errors in robotic manipulation for the precise robotic fabrication of architectural structures. Reducing errors in the assembly process through the design of geometric units expands the field of architectural robotics to designers. Through a series of assembled architectures, fabricated through both additive and subtractive manufacturing techniques, the research explores the idea of an automated system producing unit-based structures using pick-in-place robotics and digitally fabricated units.

**Keywords:** Pick-and-place · Robotic fabrication · Interlocking units · Brick stacking

# **1 Automated Assembly for Architecture and Construction**

Automation for assembly is now a common method of early-stage assembly in automotive and engineering practices. In contrast, building construction has struggled to include new technologies, such as robotics, and still relies primarily on manual labor to manufacture buildings. This is particularly the case for small buildings such as houses in North America composed of thousands of small timber or brick elements. The smallscale building industry is ready for new computer-based technologies that can automate or assist workers as part of building production. This paper presents the idea of a robotically assisted assembly of non-standard interlocking blocks.We believe this method to be supportive of the constant search for rapid production of small non-standard structures.

#### **1.1 The Current State of Construction**

Rapid assembly of unit-based structures is as old as the field of architecture. Traditional forms of manual construction use standardized sets of materials for architectural design and building. These construction and manufacturing systems rely on the precision of tradesmen, craftsmen, and laborers to design custom structures. For example, brick construction relies on standard masonry units and mortar to build both complex and simple structures requiring hundreds of hours of manual instruction and labor, for simple tasks such as block stacking and mortar setting [16]. Low-cost, high-skilled labor for modern construction is in short supply. It is clear that assistance of some type of automated system is needed for the design and building industries.

#### **1.2 Automated Brick Assembly**

Studies in automating architectural fabrication processes led to the development of robotic fabrication techniques including robotic bricklaying [22], robotic timber assembly [9] and robotic printing [7, 10]. Despite robotic bricklaying and stacking sitting at the forefront of robotic construction research, there is a need for an additional inquiry into these approaches to expand the field. Since the beginning of robotic brick construction research, methods of robotic brick assembly have attempted to automate architectural construction with standard bricks and blocks [3]. These methods include block assemblies of uniform and non-uniform walls with standard masonry units typically found on construction sites [4]. This is challenging research due to the necessity of high precision and accuracy in the robotic fabrication process, which is typically inaccessible to designers, architects, and contractors. Examples of robotic dry-stacked brick assembly demonstrate these challenges in robotic fabrication. As one of the earliest adopters of robotic architectural assembly, Gramazio Kohler's work around challenges in precision by fusing robotic pick and place operations with computational vision systems [5]. In the case of brick stacking, robotic fabrication research, which attempts to reproduce the infinite patterns and possibilities of design, is limited by the constraints of the robotic arm.

To work around problems in brick stacking with standard architectural units, architectural roboticists can find inspiration outside of using computer vision systems for robotic brick stacking in non-standard interlocking architectural units. For example, research in interlocking brick assembly, fabricated with a multiblock press using common soil, presents a low-cost, mortarless method of wall construction. In this case, interlocking blocks are assembled by hand [11]. Additionally, a system of interlocking blocks cast of concrete from digitally fabricated rubber molds has been presented as a mortarless solution to patterned, customized buildings. Also, presented is a generative system of block-making that yields location-specific, interlocking blocks ready for mortarless assembly [12].

The present work describes the idea of an automated system of unit-based wall production using pick-in-place robotics and digitally fabricated units. These systems present new opportunities for endless variations that can produce unit-based wall assemblies. With the introduction of robots into the construction site in the late 20th century, automation has discovered difficulties in automated solutions to masonry construction in bricklaying. Currently, there is an opportunity to rethink existing solutions and discover improved techniques.

#### **1.3 Robotic Construction**

The concept of on-site brick-laying robots expanded in the 1980s and 1990s with solutions like ROCCO, ESPRIT, and BRONCO [1, 6, 15]. They relied on standardized parts, rigid planning, controlled environments, and heavy-duty equipment to execute assembly tasks. In the 21st century, dry-stacked robotic bricklaying solutions by Gramazio Kohler explored new approaches to automated brick construction of complex structures, while tech companies, like the Semi-Automated Mason [3] and Hadrian X [8] worked toward fabricating conventional structures. Though many automated approaches to robotic bricklaying exist, there is insufficient use by designers of complex geometry, and automation to design and generate structures. Looking toward the future of construction, novel architectural unit designs can assist the complex tasks of automated construction.

### **1.4 Design Unitization for Architectural Construction**

Recently, digital fabrication has expanded the field of architectural design to produce complex geometries at scale using machine-driven additive and subtractive manufacturing processes. For the office of Frank Gehry, the process of tessellation, discretization, and fabrication of large-scale curvilinear structures is seen in works like the BMW pavilion, the Guggenheim Museum in Bilbao, and Zullholf Towers [13]. These projects use standardized materials and machines to create custom discrete units of architectural assembly. More recently, research projects investigating non-standard materials and their fabrication have seen a continued effort to merge bespoke found materials with digital fabrication procedures to create expressive architectural designs [14]. Though these methods make use of digitally processed designs and digital fabrication procedures, they require fabricators to assemble these structures by hand, continuing a legacy of manual assembly in construction. In the future of robotic construction, designers can leverage the capacities of additive and subtractive methods of fabrication to fine-tune robotic assembly for designers.

#### **1.5 An Opportunity for Design**

As it stands, robotic manipulators afford levels of speed, flexibility, and function through customization, automation, and precision in the design world. Yet their lack of utilization can be attributed to the need for simpler procedures that leverage the strengths of digital design and digital fabrication. If digital fabrication methods can rapidly produce custom architectural modules and robotics manipulators can repeat stacking operations, how might researchers combine these strengths for architectural designers? This paper argues that bespoke interlocking construction units can achieve precision geometry without precise movements. For precision machines, such as robotic arms, which need perfect conditions to operate, customized geometry can afford interplay between design intent and material reality. Developing design features to correct for imprecision in the environment, machines, and structures without computational vision systems is key for extending precision construction outside the context of the perfect conditions of a warehouse.

# **2 Methods**

To develop a modular system of interlocking assembly units, an automated process was used to construct architectural blocks from 3D-printed materials. The research in this paper uses a low-grade PLA and a Sindoh 3D-Wox-1 desktop FDM 3D-Printers. To generate the g-code for the 3D-Printing process, the methods employed the Sindoh STL slicer. The units were modeled using Rhinoceros 7 and the robotic path planning was conducted in Grasshopper using KUKA PRC [2]. The robotic manipulator used for the automated assembly was the Kuka Agilus KR6 R900 with a RAD two-finger parallel grippe as the end-effector. With these tools, the research created designs and generated wall assemblies of interlocking units.

From a rectangular wall profile, 2D profiles are generated and projected onto the surface of the 3D CAD model wall to create outlines for the interlocking tessellation shapes. Wall design thickness was tested at the model scale to fit within the constraints of the robotic manipulator and subsequent end-effector. The 2D shape information is developed from a v-groove grammar developed in the paper. The 2D shape information determines the size and shape of geometric design features for the physical design production of the custom architectural units. From the tessellation, three combinations of shapes are created to start, continue, and stop the layer-based assembly procedures. The decomposition of the 3D objects into 2D shape profiles leveraged an adapted version of the materialization processes as described by Sass and Oxman; although, the research process outlined for model unitization is akin to the fabrication procedures seen in the Sass and Knight paper outlining physical grammar production [17, 18].

#### **2.1 Producing 3D-Printed Units**

Design iterations were conducted to produce interlocking 3D-Printed units. The major constraint examined in the study was the implementation of v-grooves on unitized structures for their interlocking capabilities. Digital fabrication, through additive manufacturing, provided a flexible method of automated construction to create customized modules for their interlocking capabilities. The process allowed for the transformation of 2D and 3D design data to the physical environment using desktop 3D-Printers by streamlining the processes found in conventional modular construction procedures. In particular, eliminating processes of formwork and brick presses reduced the fabrication steps, while maintaining geometric accuracy.

#### **2.2 Design Unitization**

The design of individual interlocking units uses a top-down approach to discretize designed geometry into individual parts for 3D-Printing. Starting with an initial structure, designs are subdivided into labeled shapes. Using the shape grammar formalism, each labeled shape has a distinct set of attachment features applied to it that creates the interlocking geometry [18, 20, 21]. On shapes labeled *A,*subtractive v-grooves are added to the interior of the structure. On shapes labeled *C,* protruding v-features are added to the object's shape. The features on shapes labeled *A* and shapes labeled *C* form a set shape from the initial wall design before the application of the discretization process and shape rules. Shape *B* is created by applying shape addition, which merges overlapping elements of shapes *A* and shapes *B.* By adding shapes, *A, B,* and *C* together the elements for the original wall structure can extend the length of the initial wall design. From these shapes, designers can use digital fabrication to create interlocking geometric units and robotic fabrication to assemble structures (Fig. 1).

**Fig. 1.** The unitization of interlocking geometries using the shape grammar formalism

#### **2.3 Robotic Assembly**

The robotic assembly of individual units is constructed in a sequential manner starting with shapes labeled *A'* and ending with shapes labeled *C'*. Because individual units were created in the Rhinoceros' CAD environment, their digital assembly information, which contains the shape rules and subsequent assembly, is easily transferred into Grasshopper's Kuka PRC workflow. In the grasshopper environment, discrete poses of the construction units are located and mapped, from the centroid of each object to the localizing frame for the final positions in the constructed wall. Using the frames of the object, the system relies on the path-planning and trajectory optimization of the Kuka PRC plug-in to use pick-and-place manipulation to position construction units in place. Because of their interlocking capacity, once a localization frame has been established by the units for robotic assembly. Using A' shapes, the pick location of shapes *A'* and *C'* are referenced by positioning shape *B*' or *C*' in shape labeled *A'*. With the correct frame referenced in the digital environment, the method follows the robotic task-planning assembly logic of

$$(A' + (B') + C') \tag{l}$$

where *B'* can be added indefinitely to assemble the final wall structure (Fig. 2).

#### **2.4 Masonry Production of Assembly Units**

To challenge the methodology, portions of this research investigated alternative modes of fabricating custom architectural units for assembly. Rather than 3D-Printing units, to function at scales starting at the desktop and extending beyond, the research looked at concrete casting units using custom formwork for large-scale industrial robotic assembly. The concrete fabrication method employed the following: 1- thick Polystyrene Foam Sheets, Quickrete Concrete mix, and an Onsrud CNC router. The two foam sheets are glued to create stock for the mold. Due to the complex nature of the geometry, the mold was created in two parts, having to be flip milled for each unit to create both parts of the mold with the intended geometry. After 3D CAD designs are generated from the interlocking shape rules, Boolean Subtraction is used to create a negative form within the stock, and from the resulting geometry, G-Code is created for the CNC machine to mill the mold. Using the two-part mold, the concrete mixture is cast and removed to create masonry versions of the interlocking units (Figs. 3 and 4).

#### **3 Results**

From these studies, the research finds that the role of interlocking architectural units can assist processes in robotic fabrication and assembly. Through the application of the shape grammar formalism, architectural forms are unitized for construction in the physical and digital environments. Using the digital workflow provided, designs generated in CAD translate to visual programming, through specialized plugins for design implementation, robotic path-planning, and robotic control. In the end, the digital-tophysical workflow codified the assembly and fabrication sequence (Fig. 5). Furthermore, through the implementation of 3D printing, architectural designers can easily fabricate

**Fig. 2.** The robotic construction of 3D-Printed interlocking units. Where unit shape *C'* is added to unit shape *B'*

customized geometric units for interlocking assemblies. Due to the translation of shape rules into the digital environment, multiple methods of digital fabrication can be used to generate interlocking units, such as subtractive manufacturing through CNC milling. With physical and digital intelligence working together, designers can scale their unitized structure using different materials and forms.

### **3.1 Error Correction and Interlocking**

Unlike traditional methods of construction that rely on geometrically simple architectural units, this research introduces geometrically complex unitized modules for selfinterlocking structures. When assembled in the correct sequence, the geometric faces of the architectural modules allow for the registration of the architectural blocks to work with gravity to interlock and slide into place. The registration of architectural blocks into self-interlocking structures reinforces previous research in kinematic coupling and exact constraint design [19]. These research topics explore the role of contact points in precision mechanical assemblies. By aligning the design shapes rules to generate contact points through integral components of architectural modules, which constrain three degrees of freedom, we developed interlocking units for assembly. We found that there

**Fig. 3.** Design of interlocking concrete assembly using custom units *A' B' C* where units form components of a wall structure

**Fig. 4.** Design of interlocking concrete assembly using custom units *A' B' C* where units form components of a wall structure

must be a minimum of two contact points during the assembly sequence for units to interlock into the desired shape of a wall.

Since the architectural blocks contain precision geometry, that uses registration points and gravity to interlock, they eliminate imperfections in the robotic assembly processes. Nevertheless, this assembly process is reliant on low coefficients of frictional forces to enable the sliding and interlocking effects. For 3D printing, variables in printed layer heights, infill, and surface finish play key roles in the block's interlocking capabilities. In our concrete fabrication tests, we found that variables in mold surface finishes, material mixtures, and finishing processes play important roles in curating the frictional forces of masonry blocks. With controlled frictional forces, the research shows that interlocking geometries can work around imprecision in end-effector position-controlled robotic manipulators completing architectural assembly tasks. Simple robotic assembly procedures, in addition to those using computer vision sensors and tools, require high levels of precision to determine positions for assembly sequences [4]. With the methods illustrated in the paper, designers can create precision geometric units that create variable structures and designs. In other words, instead of working toward strict positioning in robotic manipulation, designers can create designs that account for inexact manipulation in the layer-based assembly process.

#### **3.2 Digital Fabrication for Bespoke Object Generation**

Incorporating methods of additive and subtractive fabrication for custom architectural fabrication aided the robotic fabrication process. Due to the highly complex nature of architectural units in the research, custom fabrication methods relied on precision machines for geometric accuracy. In the 3D-printing process, the additive method of FDM printing allowed for a streamlined method of fabricating architectural units taking the digital design information from the computer directly to the machine to create objects. Conversely, the method of subtractive manufacturing through CNC machining required the most of steps in the fabrication process. After the digital CAD design of the interlocking units, the method required stock preparation, CNC machining, mold assembly, and masonry curing for the assembly to be completed. Furthermore, the accuracy of the masonry units depended on the precision of the mold, the quality of mold assembly, and the material efficacy of the concrete mixture. Though there is a tradeoff in material size, strength, and precision, 3D printing as the first test proved successful at validating the research goal of creating interlocking assembly modules for robotic fabrication.

#### **4 Implications**

The next steps of this research seek to validate existing claims by robotically fabricating complex architectural forms, enclosures, and structures. Methods seeking to leverage layer-based assembly, while reducing the geometric complexity of objects would be positive strides in the right direction for the research. Additional research into methods of concrete CNC mold formworks, masonry material composites, and scaled robotic construction of customized geometries would extend the results of this research. On top of that, additional studies in automated design discretization of 3D CAD models would allow for a refinement of the scale, form, and shape of bespoke units for interlocking assembly systems. Discovering novel ways of fabricating non-uniform curvilinear architectural designs through discrete units and robotic assembly is an expectation of the research. Finally, the research presented in the paper opens the door for the investigations of the structural properties of bespoke architectural units. Through continued study, a distinct language for discretized architectural systems fabricated through robotic means, utilizing unique geometric architectural structures, will emerge, blending construction techniques of the past, present, and future. With these robotic techniques, designers, architects, and builders can advance the field of architectural design beyond manual practices, therefore, accelerating workflows and creating languages of robotically assembly construction.

# **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **A Parametric Wave Joint for Robotic Fabrication of Digital Stereotomy**

Wei Ye1(B) , Yingzhou Gao1, and Weiguo Xu2

<sup>1</sup> Tsinghua Shenzhen International Graduate School, Shenzhen 518055, China yew21@mails.tsinghua.edu.cn

<sup>2</sup> School of Architecture, Tsinghua University, Beijing 100084, China

**Abstract.** This paper explores the potential of digital stereotomy in combination with robotic fabrication to increase the precision and complexity of stone processing. To enable the application of these techniques in outdoor environments, modular joints designed for robotic assembly are necessary. Additionally, the cutting process must be efficient and minimize material waste. To address these challenges, this research proposes a parametric wave joint design that enables rapid cutting and straightforward assembly by a robotic system. The joint contains motion space allowing it to slide into accurate assembly position, enabling the robot to complete the assembly without requiring highly precise vision or gripper in outdoor situations. Furthermore, the wave joint design eliminates the need for milling, reducing the processing time. The paper presents a robotic arm-cutting method for this joint and conducts experiments using foam and robotic arm hotwire cutting to simulate stone cutting. The feasibility of the joint is tested through the assembly of a bent column, and finite element analysis is used to compare the stresses on two joint parts under shear force with different control parameters. The study confirms the feasibility of the wave joint design for robotic assembly and the efficiency of robotic arm cutting. The findings may inform the development of modular assemblies for robotic systems in stone processing applications.

**Keywords:** Digital stereotomy · Robotic fabrication · Wave joint · 3D assembly · Computational design

# **1 Introduction**

Stereotomy, an ancient building construction technique, has been employed for centuries, relying mainly on manual processes prior to the emergence of computer technology [3]. Digital stereotomy refers to the use of digital technologies to plan and execute the cutting of stone blocks for use in building construction. Digital stereotomy has evolved from traditional stereotomy, which involves the manual measurement and cutting of stone blocks, and has been aided by the advancement of digital technology in recent years.

While previous studies have investigated various aspects of digital stereotomy, there are still challenges that need to be addressed to enable the practical application of these techniques in outdoor environments. Modular joints designed for robotic assembly are necessary, and the cutting process must be efficient and minimize material waste. To address these challenges, this research proposes a parametric wave joint design that enables rapid cutting and straightforward assembly by a robotic system.

This study adopts a design science methodology to devise a suitable joint design for digital stereotomy. The methodology involves an extensive review of the existing literature and practical applications of digital stereotomy to identify research gaps in the current theory. Based on the identified gaps, a novel solution is proposed, and a joint design is developed alongside a parametric generative algorithm. To evaluate the feasibility of the proposed design, robotic fabrication and finite element analysis are conducted.

The joint contains motion space allowing it to slide into accurate assembly position, enabling the robot to complete the assembly without requiring highly precise vision or gripper in outdoor situations. Furthermore, the wave joint design eliminates the need for milling, reducing the processing time. The paper presents a robotic arm-cutting method for this joint and conducts experiments using foam and robotic arm hot-wire cutting to simulate stone cutting. The feasibility of the joint is tested through the assembly of a bent column, and finite element analysis is used to compare the stresses on two joint parts under shear force with different control parameters. The study confirms the feasibility of the wave joint design for robotic assembly and the efficiency of robotic arm cutting. The findings may inform the development of modular assemblies for robotic systems in stone processing applications.

#### **2 Systematic Literature Review**

This paper presents a thorough literature review of the research of modular joints, categorizing joint forms into two main groups: 2D joint and 3D joint. The 2D joint refers to joints that can be formed by the same section line extrude. Conversely, the 3D joint refers to joints that cannot be formed by a single section line extrude.

The 2D joints can be further subdivided based on their section line of curves and polylines, such as dovetail suture tab design [6] and registration groove [1], where the authors use curves to form interlocking joints and grooves. Others, such as semi-circular masonry arches [3] and helicoidal skewed arch [9, 10], use polyline joints to increase friction and squeezing forces in the same direction to achieve a steady state. The Finger joint [7], proposed in 2015, has been used in the construction of the entity to confirm the validity of the conclusion. However, 2D joints cannot provide sufficient sliding resistance for a building's lateral forces in multiple directions.

On the other hand, 3D joints, due to their different section lines in each section, appear to resist sliding in cross-section directions. This paper further categorizes these joints into multi-groove joint, wave joint, and others. For example, to create interlocking conditions, drum face [1] and groove joint [8] are often cut to form multiple grooves that resist sliding. Moreover, the most common form of study used in digital stereotomy is the wave joint, which can be cut in a single pass using a wire saw, increasing work efficiency. Examples include wave-jointed blocks [12], cone joints [13], wave joints in catenary arch [5], and osteomorphic blocks [4]. Except for those mentioned above, a universal joint [11] will build a stable three-dimensional space using robotic arms.

These different types of joints provide viable construction methods, cross-section shapes, experimental methods, and analysis methods for digital stereotomy and modular blocks. From the perspective of the fabrication system, the 2D joints can all be cut by a wire saw but cannot provide sufficient sliding resistance. The 3D multi-groove joints provide enough sliding resistance, but cannot be cut by a wire saw. The 3D wave joints can be cut by a wire saw, but the cross-section contains some weak parts that are easy to damage.

Therefore, it is important to investigate the possibilities of combining the benefits of both 2D and 3D joints, such as creating a joint that has interlocking grooves or multi-grooves that can resist sliding in all directions while being cut by a wire saw. Furthermore, the joint design should consider the strength and durability of the joint in addition to the ease of assembly, making it a reliable and efficient method for digital stereotomy (Table 1).

# **3 Design Development**

This paper seeks to explore how a complex joint can be designed to combine sliding resistance on all sides, provide motion flexibility for easy assembly, and have wire saw cutting capabilities. Creating concave and convex joints from the module itself rather than adding new material prevents the deformation of different materials from impacting durability [12]. Interlocking joints are used to limit displacement. For a groove joint with only one groove, movement is possible along the direction of the groove. Dovetail joints can only move along the direction of their groove, as the successive contact surfaces of the groove form acute angles two by two (Fig. 1a). When the successive contact surfaces of the groove form right angles two by two, the parts can move along the groove and in the direction perpendicular to the groove (Fig. 1b). When the successive contact surfaces of the groove form obtuse angles two by two, the parts can move along the range of the combined angles of the inclined angles of the two inclined contact surfaces (Fig. 1c). The described movement along the inclined contact surfaces is discussed as motion space.

#### **3.1 Motion Space**

The motion space discussed in this paper refers to the motion space between two interlocking joint parts, which facilitates easy assembly while allowing for tolerable misalignment. The motion space of the third joint (Fig. 1c) allows misalignment within a certain angle and distance (Fig. 2a). However, the motion space is also making the assembly of the third type of joint easily disintegrate, by contrast to the second type of joint (Fig. 1b) whose sliding friction surfaces are parallel to each other, allowing disassembly force direction of only along the groove and perpendicular to the groove. In order to guarantee both motion space and interlocking, it is necessary to combine the two types of joints (Fig. 1b, c). The motion space is essential for the assembly process but has a negative impact on stability. To counteract this, the lower part of the groove joint is set as inclined surfaces and the upper part as parallel surfaces (Fig. 2b). To increase the motion space while still providing parallel surfaces for interlocking, the inclined surfaces are altered to smooth curved surfaces that connect to the parallel surfaces (Fig. 2c). Such


**Table 1.** Existing joint forms

(*continued*)


**Table 1.** (*continued*)

458 W. Ye et al.

a junction allows for alignment errors during installation, as the two blocks can be slid into the correct installation position through the motion space between the joints, but after installation, the parallel faces prevent dissembling. Reduce stress concentrations This paper defines this type of joint to be a type of joint.

**Fig. 1.** The moveable direction of assembly with different joint shapes

**Fig. 2.** Misalignment that can be accommodated by joints with motion space before assembly

#### **3.2 Combination of ∧ Shape and -Shape**

 shaped joints with grooves in only one direction can be cut out by a wire saw, but such joints have the least resistance to displacement in the extended groove direction (Fig. 3a). shaped joints with two non-intersecting grooves in different directions can also be cut out by a wire saw. Though such joints can resist horizontal displacement to a certain extent, the joint material needs better shear resistance (Fig. 3b). shaped joints resist horizontal displacement best when the two grooves are perpendicular to each other and intersect, but cannot be cut out by wire saws, only by milling (Fig. 3c).

To address the low shear and tensile resistances in joints, Mousavian et al. studied interlocking joint shaping ∧ and ∨ [10]. The same two ∧ joints provide sliding resistance when they cross each other vertically and can also achieve wire saw cutting (Fig. 4a). However, in order to ensure that wire saw cutting is possible, the centroid positions of the nodes are actually not interlocked.

Another example of a joint that provides sliding resistance and can be cut by a wire saw is the wave joint developed by Weir et al. (2015), see Fig. 4b. To ensure that the joint can be formed by direct cutting with a wire saw, the profile of the joint is always formed by a straight line. The profile shows a large degree of interlocking in the area near the outside of the joints, while there is almost no interlocking in the area near the center of the joints. This also makes the joints vulnerable to damage [5].

To allow joints to be processed directly by the wire saw, we chose to vertically intersect a ∧ shaped joint with a shaped joint, with the ∧ shaped joint providing one side of the sliding resistance while the shaped joint prevents displacement from the other side (Fig. 4c). For the objective of cutting only by wire saw, this paper forms a new wave joint based on the combined joint (Fig. 4c).

**Fig. 3.** The combination joint of two shaped grooves and processing method

**Fig. 4.** Comparison of sections with different joints

#### **4 Parametric Modeling**

In order to automate and standardize the wave joint design process, the edge curves of the generated joints need to be controlled in a parametric way. The design of this joint needs to consider different parameters such as the height of the part and the ∧ part, size and curvature of the motion space. The use of the joint also needs to compare the performance of joints generated with different control variables for a given identical parameter and find the optimal joint shape to use in different situations. In addition, since the poly-surface of this wave joint is generated by lofting the edge curves, the points on the curves need to be able to correspond one to the other, so a parametric calculation method is needed to calculate the curves.

#### **4.1 Wave Curves Calculation**

Given two parts Pi and Pj, with endpoints p1 and p2 and a middle point between endpoints p0. The purpose of the wave joint is to enable wire saw processing. This requires that the profile of the cutting surface consists of two corresponding points connected by a line. In order to better make the points of the cutting surface correspond to each other, a formula for generating joint edge curves is developed in this paper. The side curve is calculated as follows:

$$\mathbf{y} = k \arctan(-|nx| + a) + \frac{k}{2}\pi,\# \tag{1}$$

When *x* = 0, *x* is located at the position p0. *k* is the occlusion depth constant, and *a* is the occlusion width constant. (*k*, > *a* 0).

For the middle curve, while the width and vertex position of the part are the same as the side curve, the part of the middle curve is shorter than the side curve due to the ∧ part needing in the other direction. So the constant *b* controlling the height of the ∧ part is added to the calculation as follows:

$$\mathbf{y} = b \text{arctan}(-|nx| + 0.9a) + (k - b)\text{arctan}(0.9a) + \frac{k}{2}\pi,\tag{2}$$

When *x* = 0, *x* is located at the position p0. *b* is the occlusion depth constant, and *n*, *a* is the occlusion width constant, (*k* − *b*) is the height constant of the ∧ shape (0 < *b* <*k*). Figure 5a shows the guide curves of the joint (setting *k* = 30, *a* = 20, *b* = 12, *n* = 1). The red line is the side curve of the joint, the green line is the middle curve of the joint. The poly-surface of the wave joint is obtained by lofting the side curves and the middle curve.

When extra space between the two parts is needed, for example, it is necessary to leave space for adhesive. The curvature of the corner can be increased by lowering the constant *n*. As *n* is lowered, the constants *k*, *b*, *a*, should be adjusted accordingly so that the concave and convex parts can be nested and occluded. For the concave part constants *k*1, *n*1, *a*1, *b*1, when *k*1, *b*1= 1.05 *k*, *b*, *n*1= 0.5 *n*, *a*<sup>1</sup> = 0.55 *a*, the side curve of the concave and convex parts are plotted as in Fig. 5b, where the blue line is the concave part and the red line is the convex part. The side curve and middle curve of the concave part are shown in Fig. 5c, the orange line is the middle curve. And Fig. 5d shows the middle curve of the concave and convex parts.

By adjusting different parameters, different joint shapes and sizes can be generated, and the joints generated by adjusting different control constants are shown in Fig. 6.

**Fig. 5.** Visualization of the equation of the edge curve of a concave part and a convex part

**Fig. 6.** Different joint shapes obtained by adjusting different constants in grasshopper. Joint shape (left) and its control constants (right)

#### **4.2 Robotic Arm Cutting**

The method proposed in this paper is a fabrication-oriented design. The reason why robotic arm stone processing is not yet widely used in construction scenarios is that robotic arm milling takes a lot of time and is too inefficient. However, the speed of robotic wire saw cutting is much faster compared to milling. For this reason, it is particularly important to design complex forms for the wire saw cut finish. The design of the wire saw cutting pattern should consider that the pattern consists of several ruled surfaces and that the integrity of the wire saw means that the path of the ruled surfaces should not be incorrectly damaged to the parts that do not need to be cut. It is also necessary to consider the shape of the tool head to avoid the wrong collision of the tool head with the object. The wave joint proposed in this paper can be done directly by wire saw cutting. Both the concave part and the convex part in this joint are cut twice by the robotic arm hot wire to be completed.

For the convex part, the guide curves of the robot arm is the edge curves of the wave joint. The two side curves and the middle curve are cut twice, as shown in Fig. 7 (left), the green lines are cut first, and then the red lines.


**Fig. 7.** Simulation of hot wire cutting process in grasshopper

The prototyped was cut using a KUKA robotic arm with a hot wire tool. A bent column prototype was fabricated and assembled, see Fig. 8.

**Fig. 8.** A bent column cut with foam hot wire to simulate stone cutting

# **5 Future Study**

This paper proposes a parametric wave joint from a processing and assembly point of view for digital stereotomy, presents its parametric generation method, and verifies its feasibility. Future research can experimentally test different parameters of these joints and verify which parameter can generate joint structures with performance that best matches the stone. In this paper, the performance of such joints with different shape opening widths was simulated. Finite element analysis was done with Abaqus CAE for two different widths of wave joint with the same other parameters, as shown in Fig. 9. From the simulation, it can be seen that when the shape is narrower, the stress concentration is mainly located at the side of the shape, while when the shape is wider, the stress concentration is mainly located at the top of the shape.

**Fig. 9.** The result of FEA on two different joints, narrow shape (left), and wide shape(right)

More finite element analysis experiments can be done in the future to further verify the feasibility of this kind of joint. In addition, in order for this parametric wave joint to be applied in real construction scenarios, its connection method should also be tested more, such as whether to add adhesive or waterproof material between the concave and convex parts, which will also bring about the change of parameters between concave and convex parts of the wave joint.

# **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Weaving Tectonics: Algorithmically Optimised Robotic FRP Weaving of Large Scale Planar Forms**

Kelton Boyter-Grant1, Zhouyang Xin1, Ding Wen Bao2, Xin Yan3, and Dan Luo1(B)

<sup>1</sup> School of Architecture, The University of Queensland, St Lucia, QLD 4072, Australia uqdluo@uq.edu.au

<sup>2</sup> School of Architecture, RMIT University, Melbourne 3000, Australia <sup>3</sup> Tsignhua University, Beijing 10084, China

**Abstract.** Steel reinforced concrete is a widely used material for constructing large spanning planar building elements due to its strength, durability, and low cost, but its environmental impact, long fabrication time, and relatively low structural performance demonstrate the need for innovation. To address these issues, this study proposes a novel design methodology and fabrication method that integrates robotic Fibre Reinforced Polymer (FRP) woven reinforcement that is optimized using a Multi-Weight Bi-directional Evolutionary Structural Optimization (MW-BESO) algorithm. The optimized FRP reinforcement is then cast in epoxy resin to produce the large scale planar building element. The methodology is evaluated through a Tabletop prototype and other small-scale rapid prototypes, which demonstrate the successes, challenges, and limitations of this approach. The study outlines the material and methodological testing conducted to assess the effectiveness of using the MW-BESO algorithm with robotic FRP weaving and describes the workflow of transforming the resulting 3D MW-BESO geometry into a 2D robotic winding path for fabrication. The research shows that this methodology has the potential to reduce the environmental impact, stimulate innovative design solutions, and streamline the fabrication of large scale building elements, providing a promising avenue for the development of sustainable and efficient construction techniques.

**Keywords:** Robotics · Winding · Fibre reinforced polymer · Architecture · Structural optimisation · Fabrication

# **1 Introduction**

Steel reinforced concrete is widely used in the architectural industry for large spanning planar building elements due to its strength, durability, and low cost (Cement Concrete & Aggregates Australia 2014). However, the environmental impact, limited design solutions, and lengthy fabrication time associated with this material are of concern. An alternative material, Fibre Reinforced Polymer (FRP) composite, is significantly lighter, reduces fabrication time, and increases durability while also providing a higher strength and stiffness to weight ratio (Bazli and Abolfazli 2020).

The following research builds on the existing discourse from the University of Stuttgart and RMIT University by exploring an innovative methodology for FPR composite elements. The methodology aims to build an workflow of for highly efficient fabrication for topologically optimized structure, by developing algorithm of translating its organic mass into continues wind path, and develop automated fabrication process. The proposed methodology is tested using the Tabletop prototype which reveals the difficulties, limitations and successes of this design and fabrication methodology. A three-stage process is developed for the tabletop fabrication; Stage One: generate the MW-BESO algorithm geometry and winding pattern; Stage Two: sort the winding pattern using an automatic sorting algorithm and Stage Three: fabrication of the Tabletop using an industrial robot arm.

#### **2 Background**

The proposed workflow builds on the current discourse of research from the University of Stuttgart and RMIT University. The University of Stuttgart focuses on architectural applications of biomimicry through robotic FRP woven pavilions (Parascho et al. 2014). RMIT University focuses on BESO algorithms to optimise structures, fabricating with 3D printing technologies (Allouzi et al. 2020). To date, the BESO algorithm and FRP robotic winding have been researched independently, however the Tabletop combines the two methodologies. The result is an innovative way to design and fabricate FRP composite structures.

Pavilion X-Form by RMIT University effectively explores the use a BESO algorithm for designing (Wen Bao et al. 2022). The project is a robotically 3D printed structural column that is designed using a BESO algorithm; creating an innovative and efficient form (Wen Bao et al. 2022). A critical component of this research is the combination of the BESO algorithm and 3D printing technologies. This methodology successfully produces an efficient workflow that optimizes fabrication of large vertical structural members that would otherwise be slow to construct and require a large volume of concrete. The Pavilion demonstrates the success of combining a BESO algorithm with 3D printing fabrication. However, the vertical nature of 3D printing limits the type of elements that can be constructed.

Robotic FRP winding technologies have been researched to create light-weight tensile structures (University of Stufttgart n.d). The advantages of using FRP composite materials with robotic fabrication are demonstrated by the ICD/ITKE Research Pavilion 2016–17 at the University of Stuttgart (Solly et al. 2018a, b). The project explores how a FRP lattice shell can create a large spanning bespoke cantilevering structure; fabricated through robotic weaving (Solly et al. 2018a, b). The Pavilion demonstrated the capabilities of the FRPs strength to weight ratio (Bazli and Abolfazli 2020). One key aspect of the research is the ability to fabricate a FRP structure that extends beyond the reach of a single industrial robot arm (Solly et al. 2018a, b). The methodology used to create larger FRP structures provides a tested precedent that indicates how FRP composite elements can replace large reinforced concrete components. The ICD/ITKE Research Pavilion 2016–17 does not explore using FRP fibre in specific locations to add reinforcement strength. Instead, it uses a uniform sweeping surface to create its high-performing organic shape. Moreover, the Pavilion does not explore uniform composite elements, an exploration that the Tabletop prototype undertakes. The proposed methodology expands upon the current research discourse by designing and fabricating a MW-BESO FRP reinforced Tabletop that is robotically fabricated.

# **3 Workflow**

The proposed workflow is in three main stages that were developed and tested for the Tabletop (Fig. 1). Stage One: generating theMW-BESO algorithm geometry and winding pattern; Stage Two: sorting the winding pattern for fabrication using an automatic sorting algorithm; and Stage Three: fabrication of the Tabletop using an industrial robot arm.

**Fig. 1. A** The generated BESO geometry, **B** the automatically sorted winding path and **C** the fabrication of the Tabletop using a robot arm.

Stage One involves the process of generating the base geometry that will then be translated into the FRP winding pattern. A MW-BESO algorithm is run with the overall dimensions, material properties and load values to create a 3D geometric component (Fig. 1A). To transform this 3D geometry into a 2D winding pattern, a script is run that creates a pattern which replicates the 3D geometry and will become the robot winding path. To replicate the structural performance in 2D, the thickness of the 3D geometry is measured and additional winding strokes are added in to mimic additional thickness in the original 3D geometry.

Stage Two focuses on turning the FRP winding pattern into a robot winding path (Fig. 1B). The FRP winding pattern processes through an automatic sorting algorithm which sorts and connects the pattern into one continuous winding path.

Stage Three is the fabrication of the Tabletop prototype which has a 1800 mm diameter and a thickness of 20 mm. The Tabletop's FRP reinforcement is woven using a robot arm to increase fabrication time and accuracy of the reinforcement's location (Fig. 1C).

#### **3.1 Topological Optimised Planar Structure**

The Multi-weight Bi-directional Evolutionary Structural Optimization (MW-BESO) algorithm, originally proposed by Yan et al. (2022), is used to generate the 3D geometry that will inform the FRP reinforcement's design. MW-BESO is an extension of the well-known Bi-directional Evolutionary Structural Optimization (BESO) method developed by Huang and Xie (2010). The BESO method aims to produce the most efficient structural layout by iteratively adding or removing materials based on Finite Elemental Analysis data. However, it can only generate a single form with the highest structural performance under a given loading condition, often without consideration of additional requirements or aesthetic preferences.

To address these limitations, the MW-BESO algorithm incorporates weighting coefficients that allow designers to adjust the impacts of different loading conditions on the optimization results. The initial model is a 3D model consisting of shell elements instead of a 2D model, meaning that the plates can resist vertical gravity loads even with a flattened shape; similar to an overhanging roof. Besides topological relationships inside designs, the shape of these rigid structures with small deformations will not be changed during the topology optimization evolution process because the material distribution is the main optimized object rather than the surface shape. FRP reinforcement is needed as it is as a gravity load-bearing solid. This approach enables the generation of multiple structural designs under the same loading conditions by varying the weighting coefficients. For the Tabletop prototype, two distinct loading cases are considered: gravity acting across the entire domain and torsion applied at the inner boundaries. Gravity tends to produce straight beams from the outer boundary to the centre, while torsion induces curved beams. By adjusting the weighting coefficients, designers can prioritise the relative importance of these two loading cases in their design concepts (Fig. 2).

**Fig. 2.** Generation of optimized plannar reinforcement via BESO

To utilise the MW-BESO algorithm's 3D solution, it must be converted into a 2D winding pattern. To achieve this, a script is run that replicates the 3D geometry's stress pattern distribution. One limitation of robotic fabrication is the reach of the robot arm. To address this limitation, the pattern is then split into 8 segments. This approach ensures that the robot arm can accurately fabricate each component and provides an example of how larger components could be fabricated on site with only one robot arm.

In the MW-BESO geometry, thicker material equates to more load stress. At this step, the pattern simply traces the MW-BESO optimised geometry and reveals the primary stress distribution. However, to incorporate the increased material distribution, an algorithm is developed to replicate the material thickness by altering the FRP stroke number based on the MW-BESO geometry.

The gap widths between the hollow sections vary in different locations based on the MW-BESO result (Fig. 3A). While using the FRP stroke to wind the pattern, the significance of the material can be demonstrated through the stroke number in different locations. Multiple strokes achieve stronger reinforcement than a single stroke. Therefore, an algorithm is developed to firstly search the affected region width of each single line. The width is utilized to determine the stroke number of each single path line. In the Tabletop, a minimum of 2 strokes and maximum of 8 strokes are demonstrated as a showcase (Fig. 3C).

**Fig. 3. A** Shows the additional material distribution, **B** shows the single stroke pattern, **C** shows the additional winding strokes.

#### **3.2 One Stroke Path Planning**

The generated MW-BESO pattern is geometrically complex. Different from the winding pattern in boundary/ridge anchor winding systems (Knippers et al. 2015) where the anchors are located at edge or geometrical ridges and the surface is defined with fibre sweeping between the anchor, MW-BESO pattern creates a surface via edge networks of indefinite complexity. Thus, additional challenges for robotic path planning are present. The challenges are:


To address these challenges, the pattern is split at each vertices to create a web of edges. To ensure all edges can be evenly traversed in a single stroke, the edges in the edge set are directional, with its reverse edges added to the edge set. According to the Euler's Theorem, adding the reverse edges into the edge set ensures that there is an exit path traversing every edge exactly once, a Eulerian path. Thus, this set of edges allows for an existing robotic winding path that traverses each edge in the initial edge set exactly twice.

To achieve this path, a path planning algorithm is developed based on depth-first search (DFS). DFS is an algorithm in graph theory for search or traverse through a graph or tree structure. Starting at a root node identified by the designer as the starting point of the winding sequence, the algorithm explores as deep as possible through the network of edges without revisiting the nodes. Once it reaches the deepest node possible where all connected nodes are visited already, it traverses all the edges to and from the connected node then recurs back to earlier branches. With this method, a Eulerian path is generated that starts with the root node, traverses each edge in the initial graph exactly twice, then returning to the root node (Fig. 4).

**Fig. 4.** Path planning with depth-first-search.

Once the sorted single stroke path is established, it is crucial to identify how the winding end effector would travel around each node, so that the fibre filament is fully anchored to the node. The strategy is to evaluate the entry and exit direction of the path, and ensure the end effector always rotates around the anchor at the angle equal or greater than 180 degrees (Fig. 5).

#### **3.3 Robotic Winding and Fabrication Process**

The Tabletop prototype is the first full scale test of the proposed workflow (Fig. 6). The fabrication process involves two main steps: robotically winding the FRP MW-BESO path and casting the FRP reinforcement in epoxy. To ensure the success of each stage, overcome limitations of the workflow and resolving unforeseen issues, rapid small scale tests were conducted (Fig. 7).

**Fig. 5. A** Demonstrates the challenge of the winding process **B** Anchor winding—travel around the anchor at the angle equal or greater than 180 degrees; **C** All fibre is anchored during continuous winding process.

**Fig. 6.** Full scale tabletop prototype.

**Fig. 7.** Exploring different prototypes using rapid small-scale tests; **A** Timber mould on baking paper with masking tape. **B** Timber mould on baking paper with FRP reinforcement. **C** Plastic mould on baking paper with FRP reinforcement. **D** Plastic mould on melamine. **E** Plastic mould on acrylic.

Winding the FRP MW-BESO sorted path reveals several issues; friction, slow winding speeds and collisions with already woven fibre. To overcome the issue of friction, a custom end effector is designed. The end effector reduces winding friction due to its cylindrical spool and the fibre feeding angle into the end effector tip. Two different tips for the end effector are tested to resolve the issues of slow winding speed and collisions. The first tip is a bent metal tube. This tip increases winding times and causes major collisions. The second tip is a straight tube with a truncated cone end. This resolved those issues but does slightly increase friction. The second limitation is the limited reach of the robot arm which is resolved by splitting the pattern into 8 segments. This method provides a workflow for a single robot fabrication process which can be advantageous over other methods which have been tested at the University of Stuttgart (Solly et al. 2018a, b). Additionally, due to the specific placement of the anchors, it is crucial that the anchors can maintain the wound FRP's shape while being easy to remove during the demoulding process. Several anchor types were tested: 3D printed anchors, screws, and nails encased in plastic tubing. To resolve the issues of leaking and difficulties with demoulding, nails encased in plastic tubing are used (Fig. 8).

**Fig. 8.** Time lapse of the winding process around an anchor; **A** Leaving the previous anchor; **B** Approaching the subsequent anchor; **C** Winding around the anchor at an angle equal or greater than 180 degrees; **D** All fibre is anchored during continuous winding process.

The casting process aims to encase the FRP reinforcement with a smooth and transparent finish. The challenges are demoulding the final Tabletop and creating a 100% transparent finish with a uniformly smooth and elegant finish. The small prototypes resulted the final Tabletop having epoxy being poured on an acrylic sheet with a plastic ring border held in place with a silicon seal (Fig. 8D, E and F). This method achieves a Tabletop that is 1800 mm by 20 mm with a smooth and transparent finish. The edge condition is further tested to create an edge that is comfortable to lean on. The final method is filleting and polishing the edge to create a rounded and transparent surface which matches the top and bottom finishes of the Tabletop.

#### **4 Discussion and Conclusion**

In conclusion, this research has introduced a novel fabrication methodology for large spanning building components using Fibre Reinforced Polymer (FRP) composite material. The success of the proposed workflow lies in its ability to accurately create complex FRP reinforcement geometry using the MW-BESO algorithm, its efficiency in fabrication time, its reduced carbon emission, and its high structural performance. The Tabletop prototype, which has a span to thickness ratio of 1800:20 mm, provides a tangible example of the applicability of this methodology to the broader architectural fabrication process.

Compared to steel reinforced concrete, the FRP composite material offers several advantages, including higher structural performance, reduced fabrication time, reduced component thickness and material usage, and lower environmental impact. Although the initial cost of the FRP composite material may be higher, it is outweighed by its reduced environmental impact and lower maintenance costs. However, the adoption of this methodology requires advancements in computational optimization to reduce the complexity in generating the FRP pattern, as well as the adoption of fabrication automation and the reduction in the cost of the FRP.

The limitations of the proposed workflow include the reach of the robotic arm, which can be addressed by fabricating the FRP path in multiple parts and stitching them together. Additionally, the workflow requires formwork for the epoxy and additional time to remove the anchors from the set FRP reinforcement. Further research is needed to reduce the amount of preparation work required for this methodology.

Once these limitations are overcome, the adoption of this workflow has the potential to transform large scale planar building elements in the architecture industry. Therefore, the proposed methodology for large scale planar building elements provides a solution to the concerns and issues faced with steel reinforced concrete fabrication, and with further development and streamlining, has the potential to revolutionize the architectural industry.

# **References**

Achintha, M.: Environmental impact and embodied energy (2016)


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Slack Pack: Fabrication System for the Dual Robotic Winding of Spatial Fiber Structures**

Harrison Hildebrandt1, Mengxi He1, Peng-An Chen1(B) , Rebeca Duque Estrada1,2, Christoph Zechmeister1,2, and Achim Menges1,2

<sup>1</sup> Institute for Computational Design and Construction, Universität Stuttgart, Keplerstrasse 11, 70174 Stuttgart, Germany

peng.an.a.chen@gmail.com

<sup>2</sup> Cluster of Excellence Integrative Computational Design and Construction for Architecture (IntCDC), Universität Stuttgart, Keplerstrasse 11, 70174 Stuttgart, Germany

**Abstract.** Advancements in technology are ushering in an era in architecture in which new design methods and tools are being developed that necessitate entirely new means of fabrication, and, inversely, novel innovations in fabrication require completely new ways of designing. Coreless filament winding is a contemporary fabrication method in which fiber reinforced polymers are robotically wound on frames. Even though research on the frame design has reached promising levels of adaptability and material efficiency, these frames limit fabrication flexibility and increase fabrication time and costs. This paper introduces *Slack Pack*, a novel fiber winding technique for the fabrication of deployable spatial structures. It eliminates the use of frames by introducing slack into the fabrication process through the controlled tensioning and un-tensioning of fibers. *Slack Pack* employs a cyber-physical fabrication system that combines a generative design workflow and a multi-agent robotic fabrication setup with a custom end effector. The proposed method is evaluated through a series of physical experiments and digital simulations, demonstrating its potential for the fabrication of spatial fiber structures.

**Keywords:** Robotic fabrication · Multi-agent · Fiber reinforced polymers · Coreless filament winding · Spatial structures

# **1 Introduction**

This research explores the development of a fabrication system that integrates digital and physical methodologies in order to produce spatial fiber structures. Spatial fiber structures are non-surface geometries, such as a space frame or 3D truss, made from fiber-reinforced polymers [FRP]. FRP are composites consisting of structural fibers, such as carbon, glass, or flax fiber, combined with a polymer matrix (Bakis et al. 2002).

Harrison Hildebrandt, Mengxi He, Peng-An Chen—Contributed equally to the content of this paper.

Coreless filament winding [CFW] is a contemporary method of fiber winding that produces geometries by robotically winding FRP around anchor points supported by a frame. Rather than use surface-based formworks typically found in filament winding, CFW achieves its desired morphological outcome through a sequence of fiber-fiber interactions and the use of discretized frames (Prado et al. 2014).

Despite the advantages of CFW, winding frames still limit geometric customizability and production flexibility, represent a high cost and time investment, restrict achievable typologies to hyperbolic, anticlastic surfaces (Bodea et al. 2021), and interrupt the designto-fabrication workflow. These limitations were the drivers behind developing a new filament winding method in which the fabrication process utilizes a multi-robot system to produce customized spatial fiber geometries **without the use of premade winding frames**.

The system proposed in this paper uses two industrial robotic arms equipped with custom hardware to wind fibers in alternating states of tension and slack in order to eliminate the reliance on premade winding frames and increase the range of achievable typologies. Through the use of slack, the resulting spatial fiber structure can be collapsed and deployed on-site via tensioning and curing. The development of this system necessitated novel geometry generation workflows, automated control and path planning algorithms, and custom hardware and fabrication methods.

# **2 Context**

#### **2.1 Coreless Filament Winding**

Developed over the past decade at the University of Stuttgart's Institute of Computational Design and Construction [ICD] and Institute of Building Structures and Structural Design [ITKE], CFW allows for the fabrication of fiber composite components without using costly, single-use formwork or mandrels by replacing surface molds with skeletalframes that can be removed after the FRP is fully cured (Solly et al. 2018). The evolution of winding frames began with the ICD/ITKE's 2012 Research Pavilion (Fig. 1a) which utilized a single temporary rigid frame to shape the full pavilion (Knippers et al. 2015). The 2013/14 Research Pavilion (Prado et al. 2014) as well as 2019's BUGA Fibre Pavilion (Fig. 1b) (Menges et al. 2022) utilized adjustable frames that could be reconfigured depending on the geometry of the component being fabricated. In 2021's Maison Fibre (Fig. 1c), components were wound in stages with different parts of the frame being added at different periods of the winding process (Gil Pérez et al. 2022).

#### **2.2 Robot Collaboration in Fiber Winding**

Multi-Robot Collaboration [MRC] consists of multiple robots working together to complete a task. Recent research has demonstrated the possibility of MRC in the fabrication of complex architectural structures without the necessity of prefabricated formwork. Spatial Metal Structures (Parascho 2019) used a dual robot system to construct a complex metal spatial structure in stages. Spatial Winding (Duque Estrada et al. 2020) and Spatial Lacing (Yang et al. 2022) utilized MRC in the fabrication of spatial fibrous structures. Such studies demonstrate the potential of multi-robot systems, particularly in CFW. The benefits of MRC in fiber winding include:

**Fig. 1.** Winding frames of: **a** ICD/ITKE Research Pavilion 2012 (Knippers et al. 2015) **b** BUGA Fiber Pavilion (Menges et al. 2022) **c** Maison Fibre (ICD and ITKE 2021)


**Fig. 2. a** Two robots share a workpiece (Prado et al. 2014) **b** 6-axis robot exchanges fiber with 2-axis gantry (Duque Estrada et al. 2020), **c** Two mobile robots manipulate fiber bobbins simultaneously (Tucker et al. 2022)

#### **2.3 Restrictions of Winding Frames for Spatial Structures**

Despite the benefits of using CFW to wind surface geometries, there are several fundamental restrictions of using a frame to wind spatial structures. These include:


• **Upwind Limitation** (Fig. 3c): In CFW, fiber is secured to an anchor by winding around the anchor's circumference. If an anchor has both a member pointing in the negative Z direction (down) as well as in the positive Z direction (up), no additional members can be wound around that anchor, except for at shallow angles, since the upward member will block the robot.

**Fig. 3. a** Fiber colliding with frame **b** Robot reachability decreasing (red areas) with the addition of members **c** Robot colliding with upward pointing members

#### **3 Methodology**

In CFW, the winding frame (Fig. 4a) is considered to be **a component in the winding process that is not part of the desired final geometry that constrains fiber at specific anchor points (Fig. 4b) until it is cured**. In the remaining context of this paper, these anchor points are referred to as nodes. These nodes as well as the resulting fiber-to-fiber interactions determine the structure's final shape and maintain fiber relationships until the resin cures (Fig. 4c). At the most abstract level, nodes are all that are required to create a fiber topology.

**Fig. 4. a** Conventional winding frame in CFW **b** Location of anchor points **c** Geometric relationships of anchor points

In a spatial structure, topology is determined by the length of members and the angles at which they connect (Fig. 4c). Thus, as long as these lengths and angles are fixed, the structure can go slack and return to its global form by being tensioned—similar to how a child alternates between states of tension and slack to create a shape in the string game Cat's Cradle (Fig. 5). In fiber winding, the ability to have members go slack means that

**Fig. 5.** Cat's cradle, children's game involving tensioning/un-tensioning of string [figure adapted from (Gupta 2002)]

nodes do not have to maintain the same position throughout the fabrication process, eliminating the need for a frame.

*Slack Pack* establishes the connection nodes as **manipulatable objects** and secures fiber around them using a technique called "localized fiber curing". Consequently, the fiber member can go slack after each wind and be placed on a two-dimensional storage plane without losing its geometric properties, such as length or interaction angles. This is achieved by utilizing a custom end-effector and a dual-robot system (Fig. 6) where one robot (NodeBot) holds two nodes, while a second robot (FiberBot) winds fiber around them. This process of retrieving a pair of nodes, winding between them, and returning them to the storage plane is repeated until a network of slack fibers is completed (Fig. 7). The result is an un-tensioned spatial structure that can be fabricated without a frame, stored and transported in a collapsed state, and deployed onsite via tensioning and onsite curing.

**Fig. 6.** Overview of Slack Pack system

This system requires the use of both custom digital and physical tools. The development of these tools was divided into two sections: **Fabrication Planning** and **Fabrication Execution**.

**Fig. 7.** Fabrication sequence of *Slack Pack* system

#### **3.1 Fabrication Planning**

Prior to the fabrication planning steps, a custom generative design tool is used to generate a fabricable global geometry using an agent-based model [ABM]. A detailed description of the geometry generation is beyond the scope of this paper.

The algorithm for the fabrication planning steps was written in IronPython—a.NET implementation of the programming language Python (.NET Foundation, n.d.) and implemented into the 3D modeling software Rhinoceros 3D through the RhinoCommon SDK and Grasshopper, a visual coding plug-in for Rhinoceros 3D (Robert McNeel and Associates 2023).

#### **3.1.1 Member Order Based on Node Z Heights**

Winding is done with a continuous fiber and follows a winding order determined by the following rules: The ending of one member must be the start of the next member. members are wound in order of the winding layer they are in (Fig. 8a). The winding layers are dictated by the z heights of each node. Multiple members within a single winding layer are wound in ascending order of the winding layers they connect to (Fig. 8b). Winding the members in ascending order ensures there are no undesired fiber collisions in the final deployment.

#### **3.1.2 Storage and Winding Position Data**

The geometry is projected onto a plane to get the necessary data for the manipulation and storage of nodes. This provides the **location of the nodes** on the storage plane, the **2D interaction angle** of members, and their **projection length** which is **scaled further** to increase the amount of slack available during storage (Fig. 9).

#### **3.1.3 Fiber Length Restriction**

Because nodes are mounted on a storage plane, when two nodes are picked up, they will be tethered to other nodes on the plane. If these tethers are not long enough, the

**Fig. 8. a** Winding layers **b** Winding order within winding layer **c** Overall winding order

**Fig. 9.** Storage position data based on geometry projection

node cannot be picked up (Fig. 10). This is algorithmically checked. Two nodes can shift along the end effector following the direction of their projected member until a valid solution is found. If no solution is found, the global geometry must be regenerated.

**Fig. 10.** Example of a fiber length restriction error

#### **3.2 Fabrication Execution**

Fabrication execution includes both the computational tools used to generate the robot control commands as well as the physical hardware that were used to carry out the fabrication. The robot that performs the winding processes is referred to as the FiberBot and the robot that manipulates the nodes is referred to as the NodeBot. The NodeBot manipulates the nodes via the custom [NodeBot] end effector. The fabrication setup is shown in Fig. 11.

**Fig. 11.** Fabrication setup

A bespoke algorithm uses the fabrication planning data to create TCP-planes to control the robots. KUKA|prc, a Grasshopper plug-in for robot control developed by the Association for Robots in Architecture (2023), takes these planes along with real world calibration data to generate the KUKA Robot Language (KRL) (KUKA AG 2023) for controlling the FiberBot and NodeBot (Fig. 12).

**Fig. 12.** Robot collaboration workflow

#### **3.2.1 FiberBot**

The generation of FiberBot toolpath planes follows the offset of each node's position (Fig. 13). The offset distance is defined by the node diameters. Once the NodeBot is positioned, the FiberBot begins winding around the nodes and the winding process is repeated according to bundle size and member thickness.

**Fig. 13.** Toolpath generation for FiberBot

#### **3.2.2 NodeBot**

The generation of NodeBot toolpath planes requires the projected node positions, a defined winding height, and a node pickup offset distance. The planes at each member's midpoint act as the reference planes for all the NodeBot sequences. The NodeBot moves in the XY plane perpendicular to the member at distances defined by the pickup offset (Fig. 14a). The movement in Z direction is defined by the winding height (Fig. 14b).

**Fig. 14.** Toolpath generation for NodeBot **a** Node pickup position at correct offset **b** Raise nodes to winding height **c** Return nodes to storage plane

#### **3.2.3 NodeBot End Effector**

In order to manipulate the nodes, the NodeBot end effector (Fig. 15a) carried out the following tasks:


**Fig. 15. a** End effector installed on the robot **b** Resistive curing and thermal imaging of node **c** Exploded view of end effector

#### **3.2.4 Robot Sequence**

The robots' choreography is determined and verified using the following methods: Both robots are calibrated to the same base plane. KUKA|prc is used to generate the KRL for each robot based off the common base coordinate system. Reachability of the planes by each robot is verified using the simulation tools in KUKA|prc. Collisions between robots during winding are eliminated because during winding, the fiber winding end effector always remains perpendicular to the node end effector. Collision avoidance of the robots during travel movements is visually verified in the KUKA|prc simulation as well as during fabrication testing.

#### **3.2.5 Onsite Deployment**

In addition to removing the frame and reducing reachability issues, winding in slack allows the completed structure to be stored and transported in a collapsed state, improving transportation efficiency (Fig. 16a). The structure can then be deployed onsite via tensioning and cured in an expandable oven (Fig. 16b). Furthermore, because the structure is fabricated in a collapsed state its final size is not limited by the work envelope of the fabrication setup, allowing for the scalability of components which is traditionally accomplished through a modular approach. In the context of this research, only unidirectional (vertical) tensioning was tested. While multi-directional tensioning could yield more complex geometries, it also requires more complex deployment strategies and equipment. It was determined that these increased equipment requirements could undermine the benefits of the frameless process. One scenario, however, where multidirectional tensioning could be achieved with minimal equipment is the deployment of fiber geometries onto existing structures, but detailed exploration of this scenario was out of the scope of this research.

**Fig. 16. a** Transportation and deployment strategy of a wound structure **b** Thermal image of prototype in deployable oven

# **4 Results**

#### **4.1 Prototypes**

Early prototypes were made using a single robot for positioning and a manual end effector (Fig. 17a) and were deployed with the robot in a separate step (Fig. 17b).

**Fig. 17. a** Semi-robotic winding of early prototype **b** Deployment of prototype structure

#### **4.2 Demonstrator**

A furniture scale structure was developed as a final demonstrator to test and evaluate the *Slack Pack* system as a whole. The demonstrator geometry was designed using the geometry generation tool and included variable member cross sections based on axial stress. All the fabrication planning steps were done using the tools discussed in this paper, including the automatic generation of FiberBot and NodeBot TCP planes (Fig. 18).

The fabrication of the demonstrator was carried out on a dual robot platform consisting of two KUKA KR210-R3100 robots mounted on linear axes (Fig. 11). The Cluster of Excellence Integrative Computational Design and Construction for Architecture (IntCDC) provided the two robots as well as the FiberBot end effector. The FRP used was

**Fig. 18.** Fully cured demonstrator with load bearing capabilities

**Fig. 19. a** verlay of FiberBot simulation **b** Overlay of NodeBot simulation

a pre-impregnated fiber comprised of Tenax-E STS40 E23 48K 3200 tex carbon fiber from Teijin Carbon and EPIKOTE Resin MGS LR 135 and EPIKURE Curing Agent MGS LH 137 with a 100:35 resin-hardener mix. The structure was wound completely on the robot setup and deployed using a gantry crane (Figs. 19 and 20).

**Fig. 20. a** Arial view of fabrication setup **b** Robot winding sequence including picking up nodes, winding fiber member, and returning nodes and wound member to storage plane.

#### 488 H. Hildebrandt et al.

The final demonstrator (Fig. 21) was comprised of 14 nodes and 36 members, used 101.3 m of fiber, and weighed 0.820 kg. It was computationally designed, completely robotically fabricated with the dual robot setup, and could support over 55 kg before deforming. For comparison, the previous prototype of a similar typology (Fig. 17), which was not algorithmically generated and only partially robotically fabricated, weighed 0.630 kg but could only support 15.7 kg. This represents only a 30% increase in weight in the demonstrator but over a 250% increase in load capacity (Fig. 22) and illustrates the value and strength of the integrated *Slack Pack* system.

**Fig. 21.** Comparison of prototypes produced with and without complete *Slack Pack* system


**Fig. 22. a** Pre-tensioned state of demonstrator **b** Deployment of the demonstrator

# **5 Discussion**

The specific contributions of this research include the development of a robotic winding process that can produce spatial geometries without the use of an external winding frame. It explores the use of multiple robotic agents that each manipulate separate material systems in order to achieve a task that neither could do independently. The direct manipulation of winding anchors and the localized curing of fiber through resistive heating are significant research outcomes that open the door for further developments, particularly as they relate to the use of slack in the winding process. The system's ability to produce collapsible structures that can then be erected onsite reduces logistical inefficiencies, helping to improve prefabrication in architecture and reduce carbon emissions from transportation.

While the methods detailed in this paper successfully present a novel CFW process, it is recognized that this research serves as a starting point and that further investigation and collaboration will be essential to fully explore the system's potential, particularly in areas such as geometry exploration, structural analysis, joint detail development, and deployment strategies. Despite having an integrated geometry generation tool, the system needs to have more reciprocity between geometry generation, structural analysis, and fabrication planning. The robot coordination should be synchronized between the two robots using a tool like KUKA.RoboTeam, a native software package provided by KUKA that allows geometric coupling of multiple robots (2023). Additionally, the hardware and material systems need to be developed further. Automated spot curing should be directly integrated into the NodeBot end effector and the effect of the rapid, localized curing on the structural capacity of the joint should be benchmarked and analyzed. In the current method, the management of slack fibers on the storage plane was not considered in detail. In order to improve slack management, alternative methods of storage have been hypothesized, such as storing each winding layer on its own storage plane or level. During prototyping, it was observed that early in the winding process slack fibers did not stick to the storage plane or to other fibers, but towards the end of the winding the earliest members began sticking to one another. The resin system in these prototypes had a gel time of approximately 10 h and the fiber volume ratio (the amount of fiber in the FRP by volume) of the composite was not accurately measured. By using a resin system with a much longer gel time and by more thoroughly exploring different fiber volume ratios, unwanted interactions between fibers in the slack state could be mitigated. Furthermore, understanding and precisely controlling the behavior of the resin system is crucial in improving the deployment and scalability of the system. Another area of future development is the behavior of nodes in the transition from the planar state to the spatial state. Currently, members are wound and stored in a 2-dimensional plane (based on projections of 3-dimensional angles). Thus, as the structure transitions from 2D to 3D the nodes rotate in a semi-uncontrolled manner. In order to improve this behavior, early prototypes of a 3D node were developed, with alignment grooves that contained fiber at custom angles. These custom grooves allow fiber members to meet at truly 3 dimensional angles and reduce or eliminate the rotation of nodes during deployment. These nodes require additional rotational axes to be added to the NodeBot end effector. Detailed integration of 3D nodes and the impact they have on the overall load carrying capacity of the structure should be explored and benchmarked in future iterations of the system. In order to thoroughly evaluate the system, more robust structural tests should be performed to evaluate how different parameters affect structural capacity. These parameters should be isolated and tested individually, beginning first with the nodes to understand the effect of node geometry and localized curing, then moving to the members to evaluate the impact of bundle size and buckling length, and lastly to the overall geometry to evaluate the effect of the global design and fabrication process. These results would better inform the fabrication process and could be used to validate the simulation and predictive tools.

# **6 Conclusion**

This research successfully developed a method of robotic fiber winding that can produce spatial typologies without the need for premade winding frames. Using multi-robot collaboration, *Slack Pack* leverages the introduction of slack into the winding process, removing the requirement for a frame and extending achievable structure sizes far past the dimensions of the fabrication envelope. The outcome is an un-tensioned spatial structure that can be stored and transported in a collapsed state before being deployed onsite via tensioning and onsite curing. This novel fabrication method necessitated a fully integrated workflow in which the geometric design was informed by the structural and fabrication constraints of the system and the fabrication methods were directly controlled by the design and computational workflows. It required custom hardware, new robot interactions, bespoke geometry generation methods, and integrated path planning tools to be developed.

The use of FRP offers opportunities to explore new fiber typologies, leverage efficient material usage, and advance additive manufacturing techniques. The research detailed in this paper contributes to these opportunities and expands the realm of possibilities for spatial fiber structures in architecture.

**Acknowledgements.** Harrison Hildebrandt, Mengxi He, and Peng-An Chen contributed equally to this research within their ITECH Master's thesis at the University of Stuttgart. The research published in this article is supported by the Deutsche Forschungsgemeinschaft (DFG) under Germany's Excellence Strategy—EXC 2120/1—390831618. The authors cordially thank the DFG. The authors also thank Prof. Dr.-Ing. Jan Knippers for his guidance and Teijin GmbH for their support.

# **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **3DCP for Complex Sites: Robotic Fabrication of Custom-Fit Slabs in Irregular Pontoons**

João Ribeiro , João Miguel Silva(B) , António Morais , Bruno Figueiredo , and Paulo J. S. Cruz

EAAD—School of Architecture, Art and Design, University of Minho, Minho, Portugal joao-mig-silva@hotmail.com

**Abstract.** This paper presents a case study on the use of 3DCP to qualify rocky pontoons with spaces for recreational use—namely sitting areas, circulation trails and fishing activities—and biodiversity protection—providing habitat and refuge for native marine species—with a focus on the challenges and opportunities associated with 3DCP prefabrication for such a complex topographical context. We first discuss the benefits and disadvantages of 3DCP over traditional methods for retrofitting strategies with the support of state-of-the-art literature review. We then present a methodology for an experimental case study, organized in three stages: (1) a photogrammetric survey and digital reconstruction of the site´s rocky landscape, (2) the creation of a tool to generate and optimize custom-fit slabs based on their location on site, intended use and role in the protection of the natural ecosystem, and (3) the robotic fabrication of these slabs through 3DCP. Finally, we present our key findings, revealing that 3DCP offers a viable and more efficient alternative for appropriating and revitalizing sites with a disorderly and highly complex topography.

**Keywords:** 3DCP · Additive Manufacturing · Photogrammetry Mapping · Robotic Fabrication · Adaptive Design · Breakwaters · Sea Life

# **1 Introduction**

Coastal erosion is a major issue in coastal areas, caused by natural or human sources. Sediments are removed and dragged away by oceanic forces such as waves, currents and tides, which causes the coastline to retreat and land to be lost. In coastal urban areas, the swell and varying tides can cause damage or losses near buildings, so maritime protection structures are used to minimize such risks. Constructing breakwaters and seawalls has been done for centuries to break and weaken ocean waves, preventing coastal erosion and protecting coastal areas from the oceans. These structures are generally characterized by being formed by hills of thousands of large natural stone or concrete elements which form voids in their geometry or gaps between them, which produce the desired effect. Fishermen, hikers and tourists often use these constructions for their work or leisure, but they can be dangerous because of their irregularity in shape and difficulty of access.

In recent years, computational design and digital fabrication have enabled architects and engineers to create complex, detailed freeform spaces that were previously unimaginable with traditional techniques. Among several known exploratory projects in this area (Craveiro et al. 2019), perhaps due to the type of material traditionally employed or by the introduction of new goals in environmental sustainability, one of the digital manufacturing methods most in focus has been 3D Concrete Printing (3DCP), namely by extrusion. Despite advantages such as eliminating the need for formwork (cutting labour costs), reduced material waste and an opportunity for mass customization, two decades later, it's still hard to find this kind of Additive Manufacturing (AM) processes used in real construction projects.

This paper proposes using 3DCP to create a system of platforms for the crowning of sea-fronts or pontoons. These slabs can provide leisure and social spaces, fishing areas, pathways with viewpoints or urban furniture for appropriation by visitors. To get the perfect fit in a complex topography, we use the advantages of concrete AM so each segment replicates the digitalized geometry of its location, supporting itself and reducing the need for mechanical connections to the site.

Besides that, the climate keeps changing. Human activities on a global scale have caused environmental problems felt worldwide. One of the most pressing and concerning issues is related to the oceans and sea life: the rising water levels and temperatures in the oceans is threatening the survival of many marine species. Coastal areas, as spaces of transition between land and sea, are also affected. They are home to important marine and wetland species whose diversity must be conserved. However, in today's rushed world, this link between the constructed and natural environments is often ignored or broken. In such sense, we believe AM processes can offer solutions of added value with regard to the creation of new micro-habitats. In our project, we configured the infill of our slabs with two main objectives in mind: increasing the strength of the prints to resist sea waves, and creating various empty pockets of space to emphasize the local marine life (Fig. 1).

**Fig. 1.** Scheme of occupation of the rocky jetties, with custom-fit prefabricated concrete components, which expand public use and marine biodiversity.

#### **2 State of the Art**

3DCP has been used before to create protect coastal structures. Experiments carried out by *WinSun* have tested the prototyping of an element similar to the "*Core-Loc®*" typology, by using 3DCP techniques for the production of an integrated formwork (Winsun 2017). Nevertheless, we believe that the role of this technology does not match the scale and number of elements required for the formation of a breakwater structure, however, its potential to offer customised solutions that guarantee the resolution of specific problems stands out.

Some examples of the use of technology for the manufacture of coral reefs are also known. One of them is the X-Reef project developed by the consortium of XtreeE (2017) and Seaboost (2017). *XtreeE* is a specialist in 3D printing on a large scale, while *Seaboost* is an expert in maintaining and creating marine habitats. These companies aimed to replicate natural coral formations, which take centuries to form. 3DCP was selected for this task since it could create the various hollows in the structure. This technology enables us to build biomimetic reefs with complex geometry and varying curvatures that wouldn't be feasible using traditional concrete fabrication. Another benefit is the cost of production, which is smaller than methods like formwork. Our approach differs in that we focus also on creating recreational spaces and public areas in hard-to-use places, instead of objects completely submerged underwater. However, we think that the space between layers of infill at the base of the modules can be also used to create colonies of microorganisms like algae or small crustaceans, as proposed by the *X-Reef* project.

Typically, when a designer wishes to construct something upon uneven terrains or rock formations, the conventional method is to flatten the area (Wibranek 2019). This strategy will be costly and damaging to the environment, as the land cannot be restored. The *SDU CREATE* research group has shown the potential of 3DCP compared to conventional construction in similar cases with the Sense-ENV project. The project aimed to create a manufacturing process where the design is adapted instantly to the surface it will be applied to (Naboni 2022). A depth sensor-equipped camera was used to track the printing surface, enabling the extrusion path to be adjusted to the surface's topography, like the terrain where the printed parts are to be placed. The team has shown it's possible to customize parts by scanning the surface to get a virtual model, then printing with a path designed for that surface. The *SDU'*s approach demonstrated the potential of 3D scanning for 3DCP, but this paper proposes a different direction. We use 3D scanning for 3DCP to enable the prefabrication of custom-fit components to real contexts.

It's also interesting for this paper's framework to consider some of the work of *Darmstadt Technical University's Digital Research Unit (DDU)*. This group acquires 3D data of irregular stones using photogrammetry and laser scanning, then creates digital twins of the rocks to integrate them in a project of assembly (Wibranek 2019). This methodology is similar to the project in this paper, the main difference being the scale of production and the new uses proposed.

# **3 Methodology**

The goal of implementing the system in the near future triggered the selection of the intervention site. After comparing a set of urban seafronts, the lighthouse area of Póvoa de Varzim (Portugal) was chosen as a case study due to its proximity to urban areas, tourist activity and fishing heritage. The intervention area was chosen based on fishing activities observed on-site, the uneven terrain, and the danger it represented in terms of access (Fig. 2).

**Fig. 2.** Localization of the case study—Póvoa de Varzim lighthouse pontoon, Oporto, Portugal.

Our workflow has six parts, as illustrated in the diagram below. First, a photogrammetric survey of the area of intervention and, second, the subsequent post-processing of the 3D model. Design issues are addressed in step three. A parametric workflow (in *Grasshopper*) was used to generate solutions based on manufacturing rules (constraints such as maximum inclination angles) and parameters set by the user (such as desired function). The two following stages concern fabrication. First, the generation of the print path for each individual slab and the addition of its internal structure. Then, in the fifth step, the manufacturing in a laboratory environment with a *Kuka KR120* robotic arm as manipulator of 3DCP process. Finally, the transport and post-tensioning in-situ to complete the final aggregation. However, the intervention's cycle will only be finished when humans use and adapt it regularly, and local fauna and flora create new micro-habitats inside (Fig. 3).

**Fig. 3.** Scheme of the working methodology.

#### **4 Digitalization**

Laser, photogrammetry and structured light scanning techniques were considered to capture the intervention area of the pontoon. To produce a suitable context mesh, photogrammetry was chosen due to its high resolution, feasibility in outdoor environments and cost-effective equipment and post-processing. A single person surveyed the area with a mid-range digital camera, taking about 120 pictures. Each image is taken by moving around the selected area of interest in circular paths, varying the distance from the rocks and the camera's angle. It is important both to avoid self-shadowing and wet conditions that create reflective surfaces, as these create defective artifacts in the final mesh. After evaluating available photogrammetry software such as *RealCapture, Meshroom* and *KIRI Engine*, we got a fast and acceptable level of quality in reconstruction through *PhotoCatch*, computing in 50 min for an area of 3.5 m2 (Fig. 4).

**Fig. 4.** Example of four pictures used for site photogrammetry (left) and generated mesh (right).

The resulting mesh is then imported into *MeshLab*, a post-processing program to clean and simplify high-poly meshes, which is necessary for the efficiency of the following design steps. We used a quadric edge collapse decimation algorithm to reduce the mesh size by 40%, with a deviation of less than 1 cm, not compromising the degree of similarity to the original rocks needed for a custom-fit print.

# **5 Generation of Custom-Fit Slabs**

The process of arriving at a geometrical definition of each slab happens through a parametric workflow that can be applied to any location in the pontoon. The system takes the delimitation of an area of interest and overlaps a rationalized grid where each cell represents one module of the discretized platform. Different topological grids were studied in order to find the best configuration that balances local support, a good interlocking behavior and a convenient geometrical outcome for fabrication in 3DCP (Fig. 5).

First, a genetic algorithm is used to find the optimal overlap of (a) an 80 × 80 cm rectangular grid (available print area and weight limits affect the size) so that all the cells can be supported by the rocks below. After, the vertical edges are broken in the middle point (b) to create an interlocking effect amongst neighbors. Finally, each strip is subdivided (c) to ensure the slope is controlled, thus guaranteeing the fabrication of each piece. This produces pieces of different sizes that do not exceed maximum dimensions allowed by our printing process.

The final grid informs the perimeter of each slab. To get a volumetric configuration, we project points along the z-axis from each module at every centimeter onto the surveyed

**Fig. 5.** Generative scheme of a platform.

mesh, then use delaunay triangulation and conservative quadratic remeshing to polish off any imperfections. Finally, this mesh is extruded to reach the intended height on site, according to its functional role. This process was chosen instead of a straightforward boolean operation between the extrusion and the surveyed mesh in order to avoid the custom-fit mesh becoming tangled, with holes and unprintable (Fig. 6).

**Fig. 6.** 3D simulation of part of the platform to be produced.

Each element is unique, serving a specific purpose in the intervention, due to satisfying four objectives: (1) conformity to the geometry of the supporting rocks that latches it in place; (2) interlock with neighboring modules; (3) meet a functional necessity between horizontal circulation, vertical circulation, and resting; (4) provide a habitat for local fauna by the variations of the structural infill pattern.

# **6 Fabrication**

A pre-production phase is needed to develop an effective production strategy for robotic manufacturing. Due to the complex and unique geometry of each slab when it contacts with the rocks of the pontoon, this phase includes tasks such as creating a print path that includes unique infill patterns and evaluating the advantages and disadvantages of the position/orientation in which the piece will be printed. After some preliminary tests, we defined that the parts should be printed laterally relative to their position in-situ.

Prototyping showed that a fabrication setup without added support can induce partial collapse when the slope induced by the supporting rock is too steep. To overcome this, similar experiments of printing highly complex geometries have reported favorable results using sand as support material (Ahmed et al. 2022). In this sense, we made some local support tests in the areas of greater instability after the deposition of the concrete, before it acquired resistance capacity. After the start of the curing process, the sand was removed, and this technique revealed not only a great increase in the success rate of the fabrication, but also minimized the geometric deviation to the digital twin.

**Fig. 7.** Section print test of one of the prototypes, using sand as support material (left) and analysis of the photogrammetry model obtained with the target geometry (right).

Figure 7, on the right side, shows an analysis performed on a photogrammetric model from a fabrication test done inside a sand enclosure. It shows that the greatest deviations (yellow) happen when the curvature moves inwards, where there is no sand to support it. However, deviation in convex curvature is minimal.

For the production of the final prototype, the first step to enable fabrication was the development of a custom slicer to process each module and extract the toolpath that guides the robotic arm equipped with the extruder. The algorithm first draws horizontal planes spaced by the height of the printed layer (10 mm). It then calculates intersections between each plane and the geometry of the slab, resulting in a collection of closed curves that define its exterior boundary. These curves are then offsetted with half the print layer thickness to ensure a correct contact with the rocks. An initial assessment is then required to evaluate the slope of the part of the piece that will be in contact with the rocks in order to determine whether a reversal in its orientation would benefit the success of the impression.

The next step in slicing is creating infill that can adapt to the three function typologies: (1) stairs, (2) circulation and (3) bench. Our infill adopts a honeycomb-like configuration for structural performance, as well as to create pocket-like habitats. Its final design is automatically readjusted to support the various functions and multiple section dimensions along the platform. In the case of the first component produced, for example, there are three variants, shown in the figure: (a) when supporting the final step of a stair; (b) when supporting the first step of a stair or bench; and (c) when supporting a flat top surface (Fig. 8).

**Fig. 8.** Different types of infill for reinforcing printed components and to ensure support for the upper layers.

Infill A, for example, is composed of three stacked chambers of different heights. Their boundaries are aligned horizontally so that we can guarantee continuity when overlapping the other infill types. Duplicate vertical lines are used to increase strength and guarantee a continuous printing path with aligned seams between layers. These are interrupted along their length to create voids for an anchoring system based on tensioned steel cables that cross the module aggregation.

Regarding the print settings, to have the best results the parameter of the layer height and thickness was tested in previous experiments with a 20 mm extrusion nozzle. These tests showed that for a print configuration with a 20 mm nozzle and a layer height of 10 mm, a layer width of 40 mm should be guaranteed throughout all of the printing process. For these large-scale prototypes, the Weber 160-1 mortar, a dry cement mixture specifically formulated for 3DCP processes, was used (Fig. 9).

**Fig. 9.** Printing process of a full-scale prototype (left) and post-tensioning aggregation scheme of two printed sections (right).

# **7 Conclusion**

We believe our proposed methodology has major benefits when dealing with sites of complex topography. Our scanning-to-fabrication setup provides a non-destructive way to intervene, while taking advantage of parametric systems to customize and modify the design to a wide range of contexts and goals. Furthermore, 3DCP is the only efficient way to produce the level of customization of prefabricated components that is needed to assure an adequate fit to such complex geometries, such as the case of irregular rocky pontoons.

However, one of the greatest obstacles to full-scale manufacturing is related to the available equipment. Until this moment, larger modules couldn't be produced because of their weight, and had to be subdivided into smaller components. This has damaging consequences to the performance in-situ, given that an increase in weight would strengthen the resistance to waves.

A more in-depth evaluation will be carried once it becomes possible to install the modules onsite and test the feasibility of the project in relation to its ability to sustain marine organisms, something yet to be proven. Further work can also be done on expanding the range in functional uses of the slabs, as well as experimenting with variations in the infill patterns to offer other more adequate habitats for marine life.

**Acknowledgments.** This work is co-funded by the European Regional Development Fund (ERDF) through the Operational Competitiveness and Internationalization Programme (COM-PETE 2020) of the Portugal 2020 Program [Project No. 47108, "SIFA"; Funding Reference: POCI-01-0247-FEDER-047108. It is co-financed by the Project Lab2PT—Landscapes, Heritage and Territory laboratory—UIDB/04509/2020 through FCT—Fundação para a Ciência e a Tecnologia and by FCT Doctoral Grant with the reference SFRH/BD/145832/2019. We are grateful to the School of Architecture, Art and Design of University of Minho for hosting and supporting the ARENA Laboratory. We are also grateful to Saint-Gobain Weber for its monitoring and supply of the material.

# **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Translucent Tectonics: Lightweight Floor Slab System Based on FDM Manufacturing**

Yizhuo Liu(B) and Hao Hua

School of Architecture, Southeast University, Nanjing, China llyyzz981210@163.com

**Abstract.** A construction method for an FDM printed floor slab system is proposed in this paper. The integration of translucent thermoplastics and additive manufacturing enables architects to develop self-explanatory tectonics that reflect the logic and construction processes. Lightweight, transparent thermoplastics such as PET and PLA can be used in 3D printing to create visual contrast to conventional solid materials. The additive manufacturing process can improve structural behavior by controlling the material distribution. Therefore, the proposed floor slab system pursues 'light and strong' via using a carefully planned toolpath for FDM printing. An entire floor is subdivided into prefabricated modular components, which are then assembled using the post-tensioning method to improve the integrity and tensile strength of the floor system. A toolpath is designed based on the internal stress of the components such that the material density reflects the structural behavior of the floor slab. The material efficiency is thereby achieved by the optimized articulation. In addition, we maximize the continuity of the printing path to enhance the printing quality and reduce the manufacturing time. This construction method is applied to the renovation of a group of industrial buildings. Prototyping experiments were carried out using translucent PLA to visualize the material distribution inside modules, manifesting the design principle of "form follows performance".

**Keywords:** Lightweight Floor Slab · 3D Printing · Structural Analysis · Post-tensioning · Toolpath Planning · Thermoplastics

# **1 Introduction**

The rapid development of digital fabrication technology in recent years has bridged the gap between computational design and material realization [1]. Among these emerging technologies, 3D printing is typically applied to the construction of irregular buildings or building components due to its capability of easily realizing free-form design [2, 3]. In addition, Fused Deposition Modelling (FDM) facilitates performance-based parametric designs by digitally materializing differentiated forms derived from optimization process [4, 5]. However, the traditional building materials, such as concrete, clay and sand, result in heavyweight building components that are difficult to either construct or demolish quickly. Therefore, a construction method for a FDM printed lightweight floor slab system is investigated in this study. The use of thermoplastic enables the floor slab system to be easily assembled and disassembled without damaging the original structure. What's more, it provides strong visual contrast to traditional building materials and is thus suitable for the renovation of old buildings.

After completing the preliminary design scheme of the building space of a group of old factories, further research of the floor slab system is carried out. The design of this FDM printed slab can be divided into three processes: the design of the global geometry, engineering the material distribution and planning the printing toolpath [6]. After the geometry of the printed modules is determined, the layer pattern of the floor slab section is designed based on a stress analysis to adapt the density of the pattern to the stress of the floor slab, and the material distribution is thereby made consistent with the force flow to improve the structural performance and material efficiency. Then, based on the layer pattern, the toolpath of the FDM printer nozzle is carefully planned. Finally, translucent PLA is used in a construction experiment to visualize the internal structural logic of the floor slab system.

# **2 Background**

This research applies additive manufacturing with lightweight composite to adaptive reuse of industrial buildings. Our approach refers to three areas: (1) the design context of factory renovation; (2) using post-tensioning method to assemble and reinforce the 3D printed thermoplastic modules; (3) the study of performance-based toolpath planning.

#### **2.1 Design Context: Renovation of a Group of Factories**

Rapid societal development has made many industrial buildings obsolete. However, these buildings have become an unintended memorial of sorts and should be given appropriate new uses. In addition, changes to the original structure need to be minimized during renovation, and the readability of the structure should be maintained [7].

In this study, four factory buildings are to be renovated. The main building structures are well preserved and the internal spaces are large. Horizontal components are used to connect the factories to strengthen the connection between different spaces. Some horizontal components extend into the factory to form floor slabs, while others are bent and transformed into stairs and walls (Fig. 1).

#### **2.2 Thermoplastic Modules and Post-Tensioning Method**

Due to the weak mechanical properties of thermoplastic, measures should be taken to improve its structural performance when applied in architecture. The most common approach is using fiber reinforcement. Mohamed [8] infused carbon fiber into polymer to realize large format additive manufacturing. Kwon [9] constructed a set of building components using fiber-reinforced plastics. However, when assembling block printed floor slabs, fiber reinforcement cannot improve the global integrity of the floor slab. Therefore, post-tensioning method, which is more often applied to concrete now, is employed in this research.

**Fig. 1.** The design scheme (the pink sections are constructed by FDM).

#### **2.3 Performance-Based Toolpath Planning**

The Italian architect Pier Luigi Nervi advocated that the beauty of structure comes from the force flow, that is, the beauty of architecture emerges from the consistency between form and structure [10, 11]. Based on this point of view, Nervi created a series of architectural works in which the structural forms follow the force flow and create strong visual impact. However, the use of traditional construction methods typically produces a homogeneous distribution of building materials and architects can only shape the external form of building components. By contrast, FDM printing can control the distribution of materials at the microscopic scale through careful design of a printing toolpath to make the microstructure of the building components conform to the mechanical logic and force flow distribution.

Some researchers have explored the use of FDM to make the material distribution conform to force flow in furniture or building components [12, 13]. Retsin [14] designed the voxel chair, in which the distribution of a continuous plastic wire is controlled by designing the movement path of a robot arm to improve the stability of the chair. Luca et al. [15] printed concrete beams using a performance-based printing toolpath design to reduce material quantities. Tam et al. [6] proposed a new toolpath planning method, where the material is deposited along paths derived from principal stress lines of a shell structure.

#### **3 Construction Method of Components**

To align the new structure with the reserved structure of the old buildings, a 4 m \* 6 m single-span column network is used for the floor slab system. Steel beams are arranged in the direction of the six-meter-long span to connect thin steel columns, such that the FDM printed floor slab only needs to cover a span of four meters in one direction. The research conducted by Liu et al. [16] shows that the tensile strength and compressive strength of 3D printed pure PLA specimens are 34.16 Mpa and 103.94 MPa, respectively. As PLA has poor tensile strength, steel cables are added to the floor slab system. A prestressed post-tensioning method is adopted to make the steel cables bear a portion of the tension induced in the floor slab system by the load (Fig. 2).

As the FDM printer has a limited range, the floor slab needs to be divided into several small modules to be printed. The divided modules will be assembled on site to

**Fig. 2.** Beam-column system supporting the floor and post-tensioning cables inside the slab (the pink sections are prefabricated by FDM printing).

**Fig. 3.** Comparison of different subdivision methods (the red dotted lines indicate the position of the post-tensioning cables)

form the slab. Figure 3 is a comparison of different division methods for the prefabricated modules. The first is row-and-column division, which has poor integrity and stability because modules can easily rotate around the steel cable when under stress. The second and third methods stagger the modules in a single direction, which represents a slight improvement over the first method. However, displacement between modules still occurs easily. The last method is a herringbone division pattern that makes each module interlock with every other module in two directions and enhances the stability of the floor slab system. Therefore, we employed the herringbone division method.

# **4 3D-Printing Toolpath Planning**

As the mechanical properties of PLA are relatively poor, the structural efficiency of the slab needs to be improved. Therefore, a force analysis of the slab is first carried out, and the internal material distribution of the floor slab is then carefully designed to reflect the structural logic of the slab. As a result, the weight of the slab is reduced and the structural strength is improved. Finally, the material distribution in the floor slab is converted into a continuous printing toolpath.

#### **4.1 Stress-Based Geometry of Slab Sections**

Due to the presence of steel beams, the floor slab only needs to cover a span of four meters in one direction, which can be regard as a combination of several simply supported sideby-side beams. Therefore, it is simplified as a simply supported beam first and Karamba for Grasshopper based on Finite Element Method [17, 18] is used to analyze the stress of the beam's longitudinal section. In the analysis diagram presented in Fig. 4a, the layer pattern must be dense in the red section under compression to create high material concentration; as the reinforcing steel cables will pass through the blue section under tension, the layer pattern can be relatively sparse.

A gradually varying quadrilateral mesh is generated following the principles mentioned above (Fig. 4b). Then diagonal lines are added to convert the quadrilateral mesh into a triangular mesh (Fig. 4c) to enhance the stability of the structure. Finally, the blue parts are passed through by the steel cable (Fig. 4d).

**Fig. 4.** Layer pattern of longitudinal section conditioned by structural analysis.

The mesh is generated as follow: the height H and width W of a given rectangle are divided into n1 and n2 line segments with lengths of h1, h2… hn1 and w1, w2… wn2, respectively. It is stipulated that hn and wn are equal difference series with tolerances of t1 and t2, respectively. The line segments are generated, and the end points are connected into a quadrilateral mesh, which is converted into a triangular mesh by adding diagonal lines. The parameters n1, n2, t1 and t2 are adjusted to change the length of the line segments, and the shape of the mesh can be thus transformed until its density is consistent with the structural analysis.

#### **4.2 Differentiate the Geometry at Different Sections**

The cross section with the layer pattern shown in Fig. 4 is placed parallel to the printing plane, however, applying the layer pattern in Fig. 4 directly to the printing path of every layer in the module will create a constant material distribution in different layers in the floor slab, which does not conform to the actual force flow and thus wastes materials. Therefore, the density of the layer pattern needs to be differentiated in each layer through parametric adjustment.

**Fig. 5.** The layer patterns are arranged in descending order with distance from the steel cable.

The sections close to the steel cable are under a similar stress to the "beam" in the thick slab and bear a higher stress than the sections in the floor slab system that are distant from the steel cable. Therefore, the layer pattern of the slab section should gradually become sparser with the increasement of the distance between the section and the steel cable to improve the material efficiency. The mesh generation process described in Sect. 4.1 can be used to vary the density of this layer pattern by adjusting the parameters n1 and n2 (the number of line segments formed by dividing the edges of the respective rectangle). The layer patterns with different densities obtained by adjusting n1 and n2 are arranged in descending order according to their distance from the steel cable, such that the density of the layer pattern increases with its serial number (Fig. 5). A channel is designed within the No. 11 pattern for the passage of the post-tensioning steel cable.

In this study, FDM printing based on fused deposition modeling (FDM) technology was employed. FDM technology is a computer-aided process that creates objects through layered deposition of fused material extruded by a nozzle installed on a digitally controlled motion system [19]. As the action of gravity precludes suspending the filament in air, support is required under each layer to prevent the upper filament from collapsing and affecting the final printing quality; Besides, there is also a limitation in the maximum angle of overhang parts [20, 21]. The sudden change in the number of line segments between two adjacent layer patterns (Fig. 5) results in a dramatic shift in the position of the line segments and therefore leads to the suspension of the upper layer filaments. Therefore, after one pattern is printed for a certain number of layers, it is not possible to directly switch to the next pattern. Instead, a transitional pattern between two patterns must be carefully planned so that the printing path does not suddenly change or suspended in air.

The following algorithm is used to generate a transitional pattern. Given a sequence an of length n, a sequence bn of length n + 1, and the number m of transitional sequences required between an and bn, the matrix Cmn can be obtained:

$$c\_{l1} = \left(1 - \frac{i}{n}\right)a\_l + \frac{i}{n} \cdot b\_1 \tag{1}$$

$$c\_{ij} = \begin{cases} \left(1 - \frac{i}{n}\right)a\_j + \frac{i}{n} \cdot b\_j, \text{if } \left| a\_j - b\_j \right| \ge \left| a\_{j-1} - b\_j \right|\\ \left(1 - \frac{i}{n}\right)a\_{j-1} + \frac{i}{n} \cdot b\_j, \text{if } \left| a\_j - b\_j \right| < \left| a\_{j-1} - b\_j \right| \end{cases} \tag{2}$$

The number sequence of the digits in the *i*th row of the matrix Cmn is the length of the line segment into which the edges of the rectangle in the transitional pattern of the *i*th layer are divided. Layered deposition of the layer patterns and the intervening transitional patterns is shown in Fig. 6.

**Fig. 6.** Layered deposition of the gradually varying layer patterns.

#### **4.3 From Geometry to 3D-Printing Toolpath**

The layer patterns generated above should be translated into a G-code file, which contains the information required for the FDM printing process, such as the coordinate of each point that the nozzle passes through, the quantity of material extruded between each pair of points, and the travel speed of the nozzle [22]. The procedure presented in Sect. 4.2 determines the coordinate of each point that the nozzle passes through. Next, these points are arranged in sequence. The distance between each pair of adjacent points is used to calculate the quantity of material extruded.

To improve printing quality and reduce printing time, the movement trajectory of the FDM printer nozzle should be drawn with as few strokes as possible to reduce repetition and interruption of the toolpath [23]. According to Euler's theorem [24], the necessary and sufficient condition that a connected undirected graph can be drawn by one stroke is that there are only zero or two odd vertices (vertices with an odd number of connected edges) in the graph. Figure 7 shows two typical layer patterns generated in this study: there are four odd vertices in the left graph and only two in the right graph. Therefore, part of the printing toolpath needs to be repeated in the left pattern, whereas the right pattern can be drawn in one stroke.

**Fig. 7.** Printing toolpaths for two typical layer patterns.

#### **4.4 Five Prototypes of Modules**

Using the herringbone division method presented in Fig. 3 to combine the generated layer patterns results in fourteen prefabricated modules (Fig. 8), and the modules are assembled by the post-tensioning method to improve the integrity and tensile strength of the slab. The fourteen modules can be divided into five categories, among which the maximum module size is 2.6 m \* 1.5 m \* 0.24 m. Although the second and third modules, as well as the fourth and fifth modules, appear similar, both pairs of modules differ in terms of the position at which the steel beam passes through and the shape of the channel created for the prestressed steel cables. These five kinds of modules can also be used at other locations as the individual spans of the floor slab system are under similar stress conditions.

**Fig. 8.** The post-tensioning assembly consists of the five types of modules.

**Fig. 9.** The slab system used in industrial building renovation.

Figure 9 shows the visual effect of using the floor slab system in the industrial building renovation project mentioned in Sect. 2.1. The translucent new part enables light to pass through the lightweight floor slab, creating a strong visual contrast to the concrete and brick of the old factories, such that people can distinguish between the building materials and construction methods of different ages.

# **5 Fabrication Experiments**

Models of 1:25 and 1:8 scale are fabricated in the laboratory. The filament sizes produced by the FDM printer should maintain the proportions of actual filaments to simulate the real printing path and the density of the layer pattern. The FDM printer in actual fabrication process produces a PLA filament width of approximately 10 mm and an individual layer height of approximately 5 mm. Therefore, the width and height of the filaments in the 1:25 and 1:8 models should float at approximately 0.4 mm \* 0.2 mm and 1.25 mm \* 0.625 mm, respectively.

# **5.1 FDM Printing the 1: 25 Scale Model**

In the FDM printing experiment at the 1:25 scale, repeated printing of each layer pattern shown in Fig. 5 causes sudden changes of the positions of some line segments in the printing path. However, the filaments produced using this ratio are extremely thin, which results in a very small partial sag deformation and therefore no discernible quality problems for the FDM printed products. The five kinds of prefabricated modules shown in Fig. 10 were printed and assembled according to the herringbone division method to yield the floor slab over two spans (Fig. 10a). Thin iron wires are used instead of steel cables to connect the modules in this experiment. Figure 10b shows the perspective below the floor slab system. As translucent PLA is used to print these modules, the material distribution and the position of the steel cables inside the floor slab can be seen clearly. In high-stress regions of the floor slab, the materials are densely distributed, reducing the floor transparency, while the floor slab is more transparent in low-stress regions, which increases the readability of the structural logic of the floor slab system.

**Fig. 10.** FDM printed 1:25 scale modules assembled with iron wires.

# **5.2 FDM Printing the 1: 8 Scale Model**

The filaments produced by the FDM printer are thicker in this scale, so sudden changes in the printing paths of two adjacent layers result in prominent deformation of unsupported suspended filaments in the upper layer, which severely affects the printing quality. Therefore, FDM printing under 1:8 scale must be performed with the transitional patterns shown in Fig. 6.

The results of a series of tests show the maximum angle of overhang per layer is approximately forty degrees, beyond which the shape of the printed objects tends to deteriorate. Consequently, this information is incorporated into the program as one of the limitations encountered while adjusting the design parameters. The aspect ratio of the filaments also impacts the printing quality. With broader width and smaller thickness, the filaments on the same layer can easily stack together, which makes the surface of the printed products uneven and leads to longer printing time. On the contrary, PLA filaments with a smaller width and larger thickness decrease the number of printing layers because the height of the desired modules remains unchanged, but it may result in an overhang angle per layer exceeding forty degrees. The results of repeated tests show that the best printing quality tends to be obtained for filaments with a height and width of 1.2 mm and 0.75 mm, respectively.

**Fig. 11.** FDM printed 1:8 scale module.

Figure 11 shows the modules printed in this experiment, within which the material distribution and steel cable can still be seen clearly. Compared to the modules printed without using the transitional pattern in the 1:25 FDM printing experiment, there are some slight changes in the vein on the surface of the 1:8 modules, but the overall material distribution trend remains the same.

# **6 Conclusion**

Many architects have long pursued the goal of capturing the beauty of structures synchronized with the force flow and materials used. The development of digital fabrication enables the use of additive manufacturing to extend realization of this goal from the scale of buildings and building components to the scale of material, whereby architects can design the internal microstructure and material distribution of building components, such as beams and floor slabs. The integration of translucent composite and additive manufacturing helps architects to develop self-explanatory tectonics which reflects the logic and construction process. This integration generates feedback between design and construction processes, which are simultaneously enhanced.

A lightweight floor slab system is designed based on FDM printing for use in an industrial building renovation project. The floor slab is divided into small modules for 3D printing, and the prefabricated modules are assembled with prestressed post-tensioning steel cables. To make the internal microstructure of the modules conform to the force flow, the advantages of 3D printing in controlling material distribution over traditional construction methods are exploited.

In this study, the layer pattern of cross-section of the floor slab is generated based on stress analysis, and the other layer patterns of different cross-sections are then adjusted according to the force flow. Therefore, the floor slab system demonstrates the interconnection between the building scale and the material scale. Translucent PLA is used in 3D printing experiments at different scales, revealing the internal structure attributed to the force flow within the floor slab. The use of lightweight materials enables the slab to be easily assembled and disassembled in a short time and has little impact on the original structures, suitable for adaptive use of industrial buildings.

In addition, there are still some future works to be done. Larger scale models (1:1 or 1:2) should be constructed and load tests will be carried out to evaluate the structural performance of the demonstrator.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **ISOMORPHISM Stylized Translations of 2D Prototype in Additive Clay Printing**

Lei Gong and Philip F. Yuan(B)

Tongji University, 1239 Siping Rd., Shanghai, China {gonglei0318,philipyuan007}@tongji.edu.cn

**Abstract.** Traditionally, the relationship between digital prototypes and fabricated entities has been explicit and stable. However, recently, the ambiguous relationship between the above is coming into our focus. This research discusses a series of methods using the industrial six-axis robot to achieve diverse stylized expressions under specific prototypes and reveals the relationship between robot-controlling and printed features in order to figure out the stylized topological relationships of digital twins. This paper begins with a consideration of the relationship between imaginary archetypes and solid entities, which leads to the possibility of hitherto unknown polysemy. Next, several methods to determine the robot motion parameters based on different style details were applied, which could examine the effect of each process according to a predefined prototype by the controlled variable method. Then, some experiments have been carried out to demonstrate the connection between robot movement and fabricated details, along with the method to generate the corresponding toolpath for specific design intent. This research shows the possibility to create diverse translations based on a targeted 2D prototype by defining robot movement, which will fundamentally improve the form-shaping capability of digital technologies.

**Keywords:** Stylization · Prototype · Ambiguity · 3D Clay Printing · Robotic Arm Control

# **1 Introduction**

The concept of the prototype in architecture has a long history and can be understood simply as an initial reference. Along with the development of typology and analytical psychology, the concept of prototypes in architecture includes spatial prototypes and gradually extends to formal prototypes. In recent years, the intervention of artificial intelligence in architectural generation syntax has made it possible to quickly generate multiple spatial solutions with a specified structure. At the same time, as a counterpart, the digital fabrication paradigm, a new digital workflow, which enables precise control over the design and generation of components, dramatically facilitates the entitlement of customized and differentiated construction components [9]. Moreover, the designer's global exposure to highly differentiated design expressions has facilitated and enforced this paradigm [3]. The contemporary interest in form customization of construction elements [6] has two primary motivations: a qualitative, design-driven desire for novel forms or an aspiration for the quantitative improvement of building performance metrics (such as structural, thermal, or acoustic) [9]. Since then, the relationship between digital prototypes and constructed entities has become ambiguous, no longer a clear one-to-one relationship but a multiplicity of meanings.

The concepts of style are two commonly used devices for design analysis and synthesis [1]. As digital prototypes and fabricated entities are two fundamental aspects of digital twins, the "black box" brought about by rapid advances in digital technologies has prevented us from understanding the formal expression capabilities of relevant technologies. By using 3D additive clay printing as an example, this research hopes to sort out the stylization relationship matrix based on the particular 2D digital prototype, which will reflect the polysemy or ambiguity between them.

# **2 Background**

#### **2.1 Additive Clay Printing**

Clay, the typically green and sustainable building material, just like other natural, pastelike materials, offers a potential reduction in the embodied CO2 that the production of buildings using conventional materials emits, in addition to its excellent insulation and fire resistance properties. Due to its excellent material properties, clay is one of the most popular building materials in history, even the oldest. Early potters utilized coil-based mimicking techniques to make clay products. This method has some consistency in the logic of Contour Crafting (CC), a layered fabrication technology of additive manufacturing that can produce unique elements with complex geometric features relatively quickly without high fixed tooling costs [9]. Contour crafting (CC) seems to be the only layered fabrication technology uniquely applicable to the construction of large structures [5].

Clay 3D printing technology has advanced tremendously in recent years. Institutions like the IAAC and digital fabrication manufacturers like WASP are utilizing robotic fabrication to extrude and deposit clay at a constant rate along a linear print path [2]. Both projects, Pylos (IAAC) and Big Delta (WASP), show the potential application of clay in a time- and cost-efficient manner for large-scale construction.

#### **2.2 Stylization**

The debate on the expressive capability of digital technology has been going on since its dawn. On the one hand, digital construction technology has made possible the production and manufacturing of complex customized components, which is unimaginable before it appears. Technological advances have brought about increased productivity and a new aesthetic paradigm revolution. On the other hand, recently, blurring the original recognizable layers has often been seen as progress in the field of technology, which does away with visible traces of the manufacturing process. However, historically, construction methods in architecture and the building industry have celebrated traces of making ranging from stone cutting to log construction [8].

The design analysis and synthesis of AM utilizing the style concept is crucial since layered 3D additive printing methods leave behind endogenous signs of manufacture. Sculptural expressions based on three-dimensional features, such as volume features and form features, and pictorial expressions based on two-dimensional features, such as material features, color features, and texture features, are two categories of stylized expressions for bespoke components.

# **2.3 Research Origin**

Digital fabrication research on the expressive impacts of AM has been quite popular. The Janus Printing Project by Harvard GSD enables multi-material printing of clay-based materials through coextrusion nozzles (Fig. 1 Left) [10]. In the Seed Stitch Wall (Fig. 1 Right) by Ronald Rael and Virginia San Fratello, the print trajectory designed visually mimics the knitting technique known as seed stitch to create a soft texture of the building [7]. The aforementioned studies, however, only seek to evaluate the effectiveness of a single approach and do not, in a broad sense, systematically resolve the ambiguity that progressively develops between them.

**Fig. 1.** Left: Janus Printing, Right: Seed Stitch Wall

Using the clay additive printing process as an example, this research aims to investigate the capability of the digital fabrication process for the physical translation of a particular 2D digital prototype under specific toolsets. It also attempts to systematically analyze the coupling relationship between the isomers and manufacturing parameters.

# **3 Methods**

Instead of using a one-way linear conceptual model process, the research workflow (Fig. 2) for this study begins with a particular 2D digital prototype, extracts the specifics of the prototype's differential characteristics, and then determines the appropriate physical construction methods using a stylized index matrix derived from empirical generalization. The matrix is created from printing tests using various methods in specific processes, such as particular process parameters, robot commands, material properties, and associated manufacturing effects. It can be claimed that there is an ambiguity between the two based on the one-to-many relationship between the textural details of the built thing and the digital prototype.

**Fig. 2.** Research workflow

Three key parts of the experiment-based study process are carried out sequentially: the formal relationship between the continuous extrusion process and clay properties, the establishment of a transformation matrix of robotic target pose, and the influence of different movement modes of the robotic arm with a fixed toolkit on form in the printing process.

#### **3.1 Formal Relationship Between Printing Process with Clay Properties**

In contrast to other clay printing methods, this study focuses on determining the robot arm motion pattern and process parameters based on a single material, a fixed kit, and a particular digital prototype in order to investigate the continuous extrusion printing method's materialization performance limits.

The method relies on a technique called paste-based extrusion, a process in which successive beads of viscous ceramic paste are deposited on a printing surface to form a 3D object in a process similar to coil pottery [10]. The deposited material compresses the still-fresh clay at lower levels during printing due to the special material qualities of the paste-based material [4], resulting in variable degrees of micro deformation. In regard to constructing details, we can make distinct features on the print surface by varying the height and width of the extruded bead with the aid of the elastic deformation characteristics of the clay, permitting a variety of print forms. As a result, the extruded bead's width and height are direct elements in forming texture. The extrusion speed is controlled by the paste pumping system, which is made up of a stepper motor and an air compressor. During printing, we must synchronize the flow of paste and the speed of the tool head.

The height and width of the paste-based print element extrudate are frequently combined with one another. We have found that the two are adversely connected within a particular range through early experiments (Fig. 3). We used a circular nozzle with an inner diameter of 2 mm for the preliminary sub-height test, so the subdivided layer height must be kept to a specific interval during printing; otherwise, insufficient interlayer adhesion will affect printing accuracy and may even cause the component to deform overall. The printed section's width and height are constrained by a number of process variables. These elements can be broadly divided into static and dynamic parameters, such

**Fig. 3.** The preliminary sub-height tests

as pumping air pressure and tool head movement velocity. Static parameters include subdivision height and tool head attitude.

Hence, starting with three common parameters—toolpath sub-height, toolpath velocity, and toolpath posture—this study will undertake a series of controlled experiments to assess the impact of process parameters on the form-shaping.

#### **3.2 Transformation Matrix of Robotic Target Pose**

Starting with static and dynamic process parameters, this experiment determines the impact of printing parameters based on various motion patterns on the shape.

**Fig. 4.** TCP rotation Transformation

Although the dynamic parameters describe the rate of change of each state, the static parameters need to characterize the tool head pose and the height of the subdivision layer using six degrees of freedom: X, Y, Z for the TCP spatial position, and A, B, and C for the TCP spatial pose. By modifying A, B, C at a fixed spatial position, the figure (Fig. 4) illustrates the spatial pose transformation of a certain target point.

$$
\begin{bmatrix} 0 & 0 \ 1 \ \mathbf{X} \\ 0 & 1 \ 0 \ \mathbf{Y} \\ -1 \ 0 \ 0 \ \mathbf{Z} \\ 0 & 0 \ 0 \ 1 \end{bmatrix} \tag{1}
$$

A fourth-rank homogeneous transformation matrix (1) should be passed through in order to convert the robot root coordinate system to the tool head TCP coordinate system. The experiment shows the rotation transformation matrix needed to guarantee that the print head nozzle orientation is vertical and downward, while the original digital prototype of the printed object determines the TCP's spatial position (X, Y, Z).

#### **3.3 Formal Effects of Different Movements with the Fixed Toolkit**

Different movement patterns of the robot are the outcome of the interaction between static and dynamic process parameters.


The experimental results (Fig. 5) show that different combinations of subdivision heights based on Z variables shape the texture of the component surface in a fairly natural way, and printing formal samples enables the construction of unique prototype patterns across distinct zones of the pattern. The component texture only generates noticeable results near the edge of the transition region, regardless of whether it is based on the A or B variables. Because the change in angular velocity between consecutive TCPs of the tool head changes more dramatically during the attitude shift than the change in linear velocity, a localized buildup of material occurs at a constant pumping rate. In terms of the dynamic process parameters, varying the tool head speed directly results in varying printed cross-section widths, which is anticipated to be used for creating particular prototype designs.

**Fig. 5.** Results of control tests

#### **4 Results**

#### **4.1 Printed Entity and Original Prototype**

Controlling these three printing factors will allow for the differentiating production of certain prototype designs, according to the controlled tests on subdivision layer height, printing speed, and printing posture. The print results of the subdivision layer height combination of 1.0–2.0 mm and the tool head speed of 20 mm/s produced the most controlled and smooth surface texture out of the four gradient combinations in the aforementioned series of testing. From the standpoint of stylization, the printed textures based on the A and B poses do not significantly differ from one another and can be categorized as belonging to the same style. As a result, just one pose will be used to create the arguments in the ensuing solid printing procedure. The prototype features are represented by the 15° rotation pose of the A variation during the solid printing process.

The selection of the initial prototype is based on the following aspects: first, the digital prototype can only influence the texture style and is not allowed to change the volume and form characteristics of the model; second, the selection of the digital prototype must reflect the generality for future application in the broader range of scenarios; therefore, a capital "X" in Rockwell font is selected for this experiment, which can represent the boundary characteristics based on horizontal, vertical and oblique directions; third, the graphic characteristics of the digital prototype should be translated into recognizable robot commands; fourth, the digital prototype must be unchanged to ensure its uniqueness.

Figure above (Fig. 6) displays the outcomes of the formal entity formation. With the interlayer order remaining intact and the entity's surface reflecting the texture of resolution, it can be seen that the texture performance of the finished entity depending on the variable of speed is the most moderate. The finished entity's texture performance based on the subdivision height variable is the best, and its surface exhibits clear signs of difference. The boundary region of the prototype mostly reflects the texture features of the completed entity based on the variable of tool head posture, with the most obvious aspects primarily in the vertical and oblique orientations.

#### **4.2 Discussion and Future Work**

The surface characteristics of the three produced entities and the features of the digital prototype demonstrate how different expressions of the same digital prototype may be created by adjusting the proper process parameters. In the era of the digital twin, this relationship shatter the stereotype of a rigid one-to-one mapping relationship between prototypes and entities, further enhancing the stylistic alternatives of particular prototypes and exhibiting the multitude of meanings in the context of digital production.

The term "isomorphism" accurately captures how different constructed things based on a particular archetype display a dialectical relationship of both diversity and identity to one another. The identity, in the topological sense, is that each entity responds to a particular prototype; the difference arises because each entity is the product of a particular production process. Although this method of exploration is presently restricted to the materialization of 2D prototypes, which can only translate flat features into surface

**Fig. 6.** Digital prototype and fabricated entities

features, the underlying technology will undoubtedly advance to higher dimensions as digital technology advances.

Further investigation of the structural and thermal qualities is not conducted in such tests, and the results are now only applied to aesthetic expression, lessening their relevance to practical concerns. In order to establish gradations that provide aesthetics and performativity throughout the volume or section of the surface, future study will expand this approach to adjust the thermal and structural qualities of clay materials.

# **5 Conclusion**

Prototypes are traditionally thought of as the starting point for designers' new shape inspiration. This study discovered that the growth of design variety can be constrained by stable and unambiguous mapping relationships. The use of stylized tools for studying polysemy of digital prototypes, particularly employing simple tool systems with single materials, is the main driving force behind this study's examination of parametric design systems and digital fabrication tools. The other two main aims of this project, which are of more interest, are to propose a process-oriented design approach and to implement techniques for manipulating robots under specified construction specifics.

Research has proven that using the same digital prototype as a base, it is possible to define the process parameters for autonomous clay printing, which would fundamentally give robotic manufacturing new formal meaning. In order to think about "process-oriented design," which demonstrates excellent implementability and spectacular challenges in further research and application, the designer's attention will shift when faced with a design prototype from a single outcome orientation to a multiple outcome orientation and back to the motion parameters and process aspects of the robot itself.

**Acknowledgements.** This research was supported by the Shanghai Municipal Science and Technology Major Project (2021SHZDZX0100), the Fundamental Research Funds for the Central Universities, the National Natural Science Foundation of China under the Joint Grant Fund (U1913603), the Collaborative Education Project of the Ministry of Education (202102560007) and the Science and Technology Innovation Action Plan for Social Development (21DZ1204500) of Shanghai Science and Technology Commission in 2021.

# **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Printing Compound-Curved Sandwich Structures with Robotic Multi-Bias Additive Manufacturing**

Eric Peterson and Bhavleen Kaur(B)

Florida International University, 11200 S.W. 8th St., Miami, FL, USA {eric.peterson,bkaur003}@fiu.edu

**Abstract.** A research team at Florida International University Robotics and Digital Fabrication Lab has developed a novel method for 3d-printing curved open grid core sandwich structures using a thermoplastic extruder mounted on a robotic arm. This print-on-print additive manufacturing (AM) method relies on the 3d modeling software Rhinoceros and its parametric software plugin Grasshopper with Kuka-Parametric Robotic Control (Kuka-PRC) to convert NURBS surfaces into multi-bias additive manufacturing (MBAM) toolpaths. While several highprofile projects including the University of Stuttgart ICD/ITKE Research Pavilions 2014–15 and 2016–17, ETH-Digital Building Technologies project Levis Ergon Chair 2018, and 3D printed chair using Robotic Hybrid Manufacturing at Institute of Advanced Architecture of Catalonia (IAAC) 2019, have previously demonstrated the feasibility of 3d printing with either MBAM or sandwich structures, this method for printing Compound-Curved Sandwich Structures with Robotic MBAM combines these methods offering the possibility to significantly reduce the weight of spanning or cantilevered surfaces by incorporating the structural logic of open grid-core sandwiches with MBAM toolpath printing. Often built with fiber reinforced plastics (FRP), sandwich structures are a common solution for thin wall construction of compound curved surfaces that require a high strength-to-weight ratio with applications including aerospace, wind energy, marine, automotive, transportation infrastructure, architecture, furniture, and sports equipment manufacturing. Typical practices for producing sandwich structures are labor intensive, involving a multi-stage process including (1) the design and fabrication of a mould, (2) the application of a surface substrate such as FRP, (3) the manual application of a light-weight grid-core material, and (4) application of a second surface substrate to complete the sandwich. There are several shortcomings to this moulded manufacturing method that affect both the formal outcome and the manufacturing process: moulds are often costly and labor intensive to build, formal geometric freedom is limited by the minimum draft angles required for successful removal from the mould, and customization and refinement of product lines can be limited by the need for moulds. While the most common material for this construction method is FRP, our proof-of-concept experiments relied on low-cost thermoplastic using a specially configured pellet extruder. While the method proved feasible for small representative examples there remain significant challenges to the successful deployment of this manufacturing method at larger scales that can only be addressed with additional research. The digital workflow includes the following steps: (1) Create a 3D digital model of the base surface in Rhino, (2) Generate toolpaths for laminar printing in Grasshopper by converting surfaces into lists of oriented points, (3) Generate the structural grid-core using the same process, (4) Orient the robot to align in the direction of the substructure geometric planes, (5) Print the grid core using MBAM toolpaths, (6) Repeat step 1 and 2 for printing the outer surface with appropriate adjustments to the extruder orientation. During the design and printing process, we encountered several challenges including selecting geometry suitable for testing, extruder orientation, calibration of the hot end and extrusion/movement speeds, and deviation between the computer model and the physical object on the build platen. Physical models varied from their digital counterparts by several millimeters due to material deformation in the extrusion and cooling process. Real-time deviation verification studies will likely improve the workflow in future studies.

**Keywords:** 3D Printing · Multi-Bias Additive Manufacturing · Robotics · Sandwich Structures · Parametric Modeling · Digital Fabrication

#### **1 Introduction**

A team of researchers at Florida International University (FIU) Robotics and Digital Fabrication Laboratory (RDF Lab) has developed a novel method for 3D printing sandwich structures using Multi-Bias Additive Manufacturing (MBAM). A major challenge for architecture and industrial manufacturing is the design and fabrication of compoundcurved thin-walled structures that are both strong and lightweight. One solution to this particular design problem is the sandwich structure (Rajpal et al. 2018). Sandwich structures rely on thin, parallel surfaces of similar material thickness and composition with an intermediary layer of hollow grid-core or hexagonal-core material (Feng et al. 2020).

The design of structural sandwiches allows for thin, lightweight structures that can resist bending moments far more efficiently than other thin-walled designs such as monocoque construction (Vinson 2018). When this assembly design is combined with compound curved topologies, the result is a surprisingly robust construction, resistant to bending moments, with a high stiffness to weight ratio (Zenkert 1995). Some common applications for lightweight high-performance compound curved surfaces include (but are not limited to) aerospace, wind energy, marine, automotive, transportation infrastructure, architecture, furniture, and sports equipment manufacturing (Vinson 2018). Recent advances in robotic AM offer a manufacturing solution to create these types of sandwich structures that may be more efficient and less labor intensive than the way they are currently manufactured.

Marine vessel manufacturing commonly uses compound curved sandwich structures made from fiber reinforced plastics (FRP). FRP sandwiches are manufactured with reuseable moulds to make stiff but lightweight hull and deck assemblies for yachts and boats (Palomba et al. 2022). Typical practices for producing FRP sandwich structures are labor intensive, involving a multi-stage process including (1) the design and fabrication of a mould, (2) the manual application of a FRP surface substrate, (3) the manual application of a light-weight foam or pattern-core material, and (4) application of a second FRP surface substrate to complete the sandwich (Al-Khazraji et al. 2023). While this is a cost-effective method for mass production, there are several shortcomings to this manufacturing method that affect both the formal outcome and the manufacturing process: moulds are often costly and labor intensive to build, formal geometric freedom is limited by the minimum draft angles required for successful removal from the mould, and customization and refinement of product lines can be limited by the reliance on moulds (Ricardou 2019). AM may offer a solution to mitigate these shortcomings (Fig. 1).

**Fig. 1.** Typical sandwich construction used in FRP marine vessel manufacturing showing dry fitting a foam core before the final FRP layer is applied (N.B.: the molded deck assembly is commonly inverted during the manufacturing process, as shown). Image from Ricardou (2019).

To make 3-dimensional objects using AM, an object or part is typically sliced into a series of 2-dimensional layers of uniform thickness that can be sequentially extruded relative to a fixed horizontal reference plane (this will be referred to as laminar printing). While laminar printing is an efficient method for making certain types of geometry it is an ineffective method for printing robust, thin-walled curved surfaces (Gardner et al. 2018). Meanwhile, MBAM offers a method of printing that relies on toolpaths that trace 3-dimensional curved paths through a part with a dynamically changeable reference plane (Fig. 2).

# **2 Methods**

Our method for printing compound curved sandwich structures relied on the 3D modeling software Rhinoceros to create NURBS surface geometry, the visual scripting software Grasshopper to transform that geometry into toolpaths, and the robotic programming plug-in KukaPRC to generate movement programs. To ensure faster and cleaner prints,

**Fig. 2.** Laminar printing traces sequential 2D toolpaths (yellow) parallel to a fixed horizontal reference plane. MBAM printing traces 3D toolpaths (red) with dynamically changing reference planes such that the extruder remains perpendicular to the surface curvature normals.

we created continuous spiral toolpaths for each of three sequential movement programs: (1) a compound curved base surface printed using a laminar extrusion method, (2) a MBAM grid-pattern core, and (3) a horizontally constrained MBAM outer surface (Fig. 3).

While speed, temperature, and extrusion rates were controlled using a purpose-built manual switch box, digital I/O (Input/Output) information was included in each toolpath to turn on and off the extruder at the beginning and end of each movement program.

The fabrication setup consisted of a 6-Axis Kuka KR10-R1100 robotic arm with a non-integrated Massive Dimension pellet extruder featuring a 1mm nozzle printing on a 600 × 600 mm build platen. A custom switch box controlled basic extruder functions and settings such as extrusion rate, hot end temperature, and cooling fan speed. Using these parameters for extrusion, the minimum and maximum layer thickness achieved during the experiments ranged from 0.5mm to 3mm. Demonstration prints used transparent PLA pellets for laminar printed surface substrates, while pattern core toolpaths were printed in black.

Stage one of the printing process consisted of a simple laminar print toolpath which will be referred to as the base surface. The extruder nozzle was oriented perpendicular to the build platen and a compound curved surface was extruded ensuring that the geometry maintained only modest overhangs to avoid heat related deformation or collapse. This base surface described and established the geometry upon which subsequent toolpaths were offset and extruded (see Fig. 4).

Stage two of the printing process requires an understanding of tool center point (TCP): the orientation of the nozzle in relation to the printed part. For the grid-core printing, we mapped desired toolpath points onto a hypothetical surface offset from the base surface at a distance equal to the diameter of the nozzle (1 mm). The points were interpolated into a spiral tool path with the TCP maintaining an orientation perpendicular to the normal of the curvature of the offset surface (see Fig. 5). In order to ensure proper adhesion of the grid core to the base surface we used KukaPRC to generate

**Fig. 3 a** The Base Layer Toolpath script shows a simple laminar toolpath printed with the extruder oriented perpendicular to the build platen. **b** The Gridcore Toolpath script shows a convex grid pattern printed with the extruder oriented perpendicular to a plane describing the average of the curvature normals of the Base surface. **c** The Final Layer Toolpath script shows a laminar toolpath printed with the extruder oriented perpendicular to a plane describing the average of the curvature normals of the Gridcore and Base surface. **d** The Grasshopper script shows the formulation for three sequential toolpaths labeled Base, Grid Core, and Final Layer describing an integrated workflow for generating the MBAM toolpaths.

nominally slower linear movements. Cooling fans were deployed to prevent overheating and potential deformation of the base surface.

Stage three of the printing process used a similar offset 1mm from the interpolated surface of the grid core with a toolpath similar to the base surface. However, similar to stage two, the movement program oriented the TCP of the extruder perpendicular to the normal of the curvature of the offset surface. So, while it traced a laminar toolpath, its movement parameters maintained an orientation perpendicular to both the base surface and the grid core (see Fig. 6).

**Fig. 4.** Stage one: a simple laminar print of the base surface where the extruder TCP is oriented vertical to the build platen.

**Fig. 5.** Stage 2: the grid core is printed using vertical and horizontal MBAM toolpaths where the TCP is oriented perpendicular to the normal of the curvature of the base surface.

# **3 Results**

Initial testing of this method for printing open grid-core sandwich structures appears promising. The MBAM method and toolpaths shown in this paper cannot be produced with conventional planar slicers. Using the computational workflow and fabrication process described above we were able to successfully print a series of sandwich structures that demonstrate the feasibility of printing lightweight compound curved parts that may prove to be more robust than conventionally printed parts and conventionally

**Fig. 6.** Stage 3: the final layer is printed using MBAM toolpaths where the TCP is oriented perpendicular to the normal of the curvature of the base surface.

manufactured sandwich structures. While testing will be required to verify these results, performance analysis remains outside the scope of this investigation.

The method we developed for MBAM printing of sandwich structures relies on several factors at the intersection of computational design and fabrication testing. First, we had to determine the shortest continuous toolpath to reduce the overall print time: we found that a spiral toolpath proved the most effective. It was also observed that a strategic combination of laminar toolpath for the base surface and multi-bias toolpaths for grid core and outer shell, allowed for better cross-sectional strength between the layers. Second, we had to determine the orientation of the extruder perpendicular to the curvature normals of the base surface while also avoiding collisions between the extruder, the part, and the robot arm. To avoid these collisions, instead of directly printing the gridcore normal to the base surface, we introduced an additional rotation of the TCP (see Figs. 4 and 5). The print time of grid-core layers, was also increased to allow for PLA to slowly deposit and adhere better to the base surface. Finally, the motion planning of the extrusion also depends on the hardware and the design of end-effectors to determine feasibility of the toolpaths. We calibrated the movement speed of the robot with both extrusion rate and hot end temperature using a manually controlled switchbox in order to maintain predictable material extrusion results. We found that modest changes in these three variables led to dramatically different results.

### **4 Discussion**

There are a series of issues that we resolved in order to develop this method for MBAM printing of sandwich structures. In this section we will discuss how we resolved the critical challenges we encountered and suggest areas of future fabrication research that will be needed in order to refine this MBAM printing method.

Our printing method proved especially successful for broad areas of modestly inflected compound curved surfaces. On modestly convex surfaces the orientation of the extruder perpendicular to curvature normals was less critical than anticipated. On these surfaces we found that a single reference plane was sufficient for extruding the Stage Two grid core material. However, relying on a single reference plane for extruding the Stage Three final surface was less successful, resulting in a part with less predictable surface characteristics than the Stage One base surface. Additional testing will be required to determine the critical limits to TCP perpendicularity relative to convex and concave surfaces.

On more radically curved or folded surface inflection points the printing method was especially problematic with our equipment. We found that consistent movement speed of the robot during reorientation to a folded surface did not always correspond with consistent movement speed of the nozzle. This led to excessive material extrusion on outside corners (see Fig. 7). This issue can be easily corrected: our rather primitive setup used a constant extrusion rate relative to the movement speed of the robot. With a more fully integrated extrusion tool it is quite simple to add g-code that links the extrusion rate relative to the motion speed of the tool TCP along a toolpath. However, additional testing will be required to fine-tune the behavior of the tool on outside corners of folded surfaces.

**Fig. 7.** Initial Stage One and Stage Two testing with modestly curved surfaces.

Unsurprisingly, we found that extruded material behaved differently at various stages of the MBAM printing process. The temperature, extrusion rate, and movement speed needed to be adjusted based on the orientation of the extruder and the direction of the extrusion toolpath (horizontal vs. vertical). Moreover, the effects of gravity on material adhesion, slumping, sagging, and deformation remained less predictable in Stage Two and Stage Three of the printing process. Our ability to manually fine-tune extrusion rate, temperature, and fan speed proved to be critical for achieving acceptable results. Extensive fabrication testing will be required in order to establish baseline settings for various tool parameters.

Variability in the actual geometry of the printed part relative to the 3D model resulted in inconsistent printing behavior. Even modest slumping in the base surface during the cooling process led to irregular spacing between the extruder nozzle and the base surface upon which new toolpaths were traced. This variability due to slumping produced knockon effects which required us to manually reposition the build platen to close the gap between the nozzle and the surface. At other points, the nozzle was too close to the surface resulting in secondary heating of the base surface and subsequent deformations (see Fig. 8). Additional testing and experience working with specific geometry, materials, and settings can address these issues.

**Fig. 8.** Initial Stage Two and Stage Three testing with modestly curved surfaces.

The reach of our robotic arm and the configuration of our extruder proved to be serious limitations to our ability to test various parts at different scales. In particular, the size of our pellet extruder limited our movement and orientation options due to self-collision. The mesh model of the extruder we used in Kuka PRC failed to register collisions with the robotic arm during simulations. Therefore, we had to test our movement programs extensively to avoid damaging our equipment. An ideal extruder mounting point would be further from the nozzle and feature less material above the wrist resulting in fewer opportunities for collisions with the robotic arm. Additionally, a filament extruder would likely be smaller in size and would not be subject to some of the gravity related feed errors that we encountered with our pellet extruder.

In conclusion, our method of MBAM printing grid core sandwich structures suggests several promising practical applications in 3D printing including furniture pieces and components, marine vessel deck structures and accessories, architectural brackets and struts, add-on grid core extrusions on existing curved surfaces, and MBAM toolpaths over sacrificial formworks. It is our hope that additional research will lead to full-scale prototypes that can be tested for strength in a lab and field tested for durability. Further, research with other thermoplastic, thermoplastic composites, and fiber reinforced thermoset materials can also be explored to overcome limitations of adhesion between layers and achieve higher strength to weight ratios. The results of this initial research are promising, contributing useful new knowledge to existing research on MBAM and non-planar 3D printing with novel toolpath configurations that extend the utility of 3D printing beyond conventional slicing methods.

#### **References**


**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

# **Author Index**

#### **A**

Asan, Galo Moncayo 295

#### **B**

Balzar, Mark 295 Balzar, Zeynep Aksöz 295 Bao, Ding Wen 466 Bao, Hanzhe 408 Bogosian, Biayna 356 Boyter-Grant, Kelton 466 Braham, William W. 284 Burry, Jane 235 Buš, Peter 368

#### **C**

Chen, Difan 211 Chen, Haoyi 379 Chen, Min 139 Chen, Peng-An 476 Chen, Zexin 346 Cheng, Xihan 418 Ciorciari, Joseph 235 Cruz, Paulo J. S. 492

#### **D**

del Campo, Matias 3 Deng, Qiaoming 11, 125 Ding, Chuhua 78 Duan, Xuexin 153

**E** Estrada, Rebeca Duque 476

**F** Figueiredo, Bruno 492

#### **G**

Gao, Yingzhou 454 Gong, Lei 515 Gong, Pixin 273 González, Alberto Fernandez 102

#### **H**

Han, Dongchen 346 He, Mengxi 476 He, Xiaoxu 90 Hildebrandt, Harrison 476 Htet Kyaw, Alexander 331 Hu, Kai 11, 125 Hua, Hao 503 Huang, Chenyu 273 Huang, Jeffrey 248 Huang, Kaijie 211 Huang, Songhua 139 Huang, Xiaoran 273

#### **J**

Jasienski, Jean-Philippe 65 Johanes, Mikhael 248

#### **K**

Kafaei, Mohsen 235 Karastathi, Nikoletta 102 Kaur, Bhavleen 526 Kim, Frederick Chando 248 Knippers, Jan 51 Krtschil, Anna 51

#### **L**

Latifi, Mehrnoush 235 Li, Han 11 Li, Jiaxing 418 Li, Li 180 Li, Linxue 284 Li, Wenjing 211, 315 Li, Zao 35 Lin, Dandan 346 Liu, Fuyuan 139 Liu, Mengman 78 Liu, Yizhuo 503 Liu, Yubo 11, 125 Liu, Zidong 408 Locatelli, Daniel 51 Lok, Leslie 331

© The Editor(s) (if applicable) and The Author(s) 2024 C. Yan et al. (Eds.): CDRF 2023, *Phygital Intelligence*, pp. 537–538, 2024. https://doi.org/10.1007/978-981-99-8405-3

Lu, Yihan 180 Luo, Dan 190, 466 Lyu, Zexi 35

#### **M**

Ma, Haoran 24 Ma, Nan 284 Makvandi, Mehdi 315 Manninger, Sandra 3 Mehdizadeh, Samim 395 Melnyk, Virginia Ellyn 433 Menges, Achim 51, 476 Morais, António 492

#### **N**

Nasiri, Simin 114

#### **O**

Orozco, Luis 51

#### **P**

Pasquero, Claudia 379 Peterson, Eric 356, 526

#### **Q**

Qiu, Waishan 211

#### **R**

Rasneur, Sylvain 65 Ribeiro, João 492

#### **S**

Sahin, Ekin Sila 51 Sampson III, Myles B. 443 Sass, Larry 443 Schumacher, Patrik 153 Shan, Yunxiang 284 Silva, João Miguel 492 Spencer, Lawson 331 Su, Yunsheng 418 Sun, Lujie 346 Sun, Mingyu 90 Sun, Zhuoyang 315

#### **T**

Tessmann, Oliver 395

Tong, Ziyu 225

#### **V**

Vassigh, Shahin 356

#### **W**

Wagner, Hans Jakob 51 Wang, Hui 78 Wang, Lizhe 139 Wang, Shiliang 273 Wang, Sijia 211 Wang, Sining 346 Wang, Tianjun 190 Weimin, Zhuang 260 Wu, Zijing 35

#### **X**

Xiang, Zhouyi 139 Xin, Zhouyang 466 Xinting, Gao 260 Xu, Weiguo 454 Xu, Xinhui 315

#### **Y**

Yan, Xin 466 Yang, Chunxia 201 Ye, Wei 454 Yuan, Philip F. 305, 315, 515 Yueyang, Wang 305

#### **Z**

Zang, Beibei 190 Zastavni, Denis 65 Zechmeister, Christoph 476 Zeng, Wentao 163 Zhan, Ming 201 Zhang, Hanyi 163 Zhang, Qi 284 Zhang, Siyuan 211 Zhang, Yaoping 180 Zhang, Zhilan 125 Zhang, Zhouyu 180 Zhao, Yongqin 211 Zheng, Hang 225 Zheng, Hao 24 Zivkovic, Sasa 331